+- Smart Home Automation Forum (https://www.kincony.com/forum)
+-- Forum: Technical Support (https://www.kincony.com/forum/forumdisplay.php?fid=20)
+--- Forum: KinCony AS (https://www.kincony.com/forum/forumdisplay.php?fid=64)
+--- Thread: how to use AS ESP32-S3 voice assistant in home assistant (/showthread.php?tid=6904)
Pages:
1
2
how to use AS ESP32-S3 voice assistant in home assistant - admin - 10-22-2024
Set Up Voice Assist Pipeline
To set up voice assistant, we would need two components to create the Voice Assist pipeline in Home Assistant.
Whisper: For speech-to-text
Piper: For text-to-speech
1: Whisper Addon
2: Piper Addon
3: And finally the entire Voice Assist pipeline.
You don't need to set up the Wake Word addon, as we will use the on-device Wake Word detection with the Micro Wake Word framework
So make sure to watch it and set up the Voice Assist pipeline
ESPHome - add device - create new device called AS:
copy and paste KinCony AS yaml file:
Code:
esphome:
name: as
friendly_name: AS
platformio_options:
board_build.flash_mode: dio
on_boot:
- light.turn_on:
id: led_ww
blue: 100%
brightness: 60%
effect: fast pulse
esp32:
board: esp32-s3-devkitc-1
framework:
type: esp-idf
sdkconfig_options:
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
CONFIG_AUDIO_BOARD_CUSTOM: "y"
psram:
mode: octal # quad for N8R2 and octal for N16R8
speed: 80MHz
# Enable logging
logger:
hardware_uart: USB_SERIAL_JTAG
# Enable Home Assistant API
api:
encryption:
key: "TFpb+pBAvQIS1MVwaA7EoJ2DkpWE+79UvVro7yMyGdU="
on_client_connected:
then:
- delay: 50ms
- light.turn_off: led_ww
- micro_wake_word.start:
on_client_disconnected:
then:
- voice_assistant.stop:
ota:
- platform: esphome
password: "1245211a05eef56614a2ef5a3f3e971c"
wifi:
ssid: "KinCony"
password: "a12345678"
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Esp32-S3-Wake-Word"
password: "LJfUrdJk3svP"
captive_portal:
button:
- platform: restart
name: "Restart"
id: but_rest
switch:
- platform: template
id: mute
name: mute
optimistic: true
on_turn_on:
- micro_wake_word.stop:
- voice_assistant.stop:
- light.turn_on:
id: led_ww
red: 100%
green: 0%
blue: 0%
brightness: 60%
effect: fast pulse
- delay: 2s
- light.turn_off:
id: led_ww
- light.turn_on:
id: led_ww
red: 100%
green: 0%
blue: 0%
brightness: 30%
on_turn_off:
- micro_wake_word.start:
- light.turn_on:
id: led_ww
red: 0%
green: 100%
blue: 0%
brightness: 60%
effect: fast pulse
- delay: 2s
- light.turn_off:
id: led_ww
light:
- platform: esp32_rmt_led_strip
id: led_ww
rgb_order: GRB
pin: GPIO16
num_leds: 1
rmt_channel: 0
chipset: ws2812
name: "on board light"
effects:
- pulse:
- pulse:
name: "Fast Pulse"
transition_length: 0.5s
update_interval: 0.5s
min_brightness: 0%
max_brightness: 100%
# Audio and Voice Assistant Config
i2s_audio:
- id: i2s_in
i2s_lrclk_pin: GPIO3 #WS
i2s_bclk_pin: GPIO2 #SCK
- id: i2s_speaker
i2s_lrclk_pin: GPIO6 #LRC
i2s_bclk_pin: GPIO7 #BLCK
microphone:
- platform: i2s_audio
id: va_mic
adc_type: external
i2s_din_pin: GPIO4 #SD pin on the INMP441
channel: left
pdm: false
i2s_audio_id: i2s_in
bits_per_sample: 32 bit
speaker:
platform: i2s_audio
id: va_speaker
i2s_audio_id: i2s_speaker
dac_type: external
i2s_dout_pin: GPIO8 # DIN Pin of the MAX98357A Audio Amplifier
channel: mono
micro_wake_word:
on_wake_word_detected:
# then:
- voice_assistant.start:
wake_word: !lambda return wake_word;
- light.turn_on:
id: led_ww
red: 30%
green: 30%
blue: 70%
brightness: 60%
effect: fast pulse
models:
- model: hey_jarvis
voice_assistant:
id: va
microphone: va_mic
noise_suppression_level: 2.0
volume_multiplier: 4.0
speaker: va_speaker
on_stt_end:
then:
- light.turn_off: led_ww
on_error:
- micro_wake_word.start:
on_end:
then:
- light.turn_off: led_ww
- wait_until:
not:
voice_assistant.is_running:
- micro_wake_word.start: yaml file download from: https://www.kincony.com/forum/showthread.php?tid=6890
now connect AS to your Server16's USB port (raspberry pi or your own server)
click right top windows "install" button, then will show you install way, chose "Plug into the computer running ESPHome Dashboard" item
click "USB=JTAG" option that detected.
ESPHome begin compile and download firmware:
now you will find the AS device discovered on "setting" - "device webpage". Just click "CONFIGURE" button, assign a room to the device.
AS is ready.
if you speak "hey jarvis" command, voice assistant will show "listening", then you can say, such as "turn on light", "turn off light". just say the name of your device.
RE: how to use AS ESP32-S3 voice assistant in home assistant - biofects - 12-04-2024
So far so good. I have everything up and running. I'm looking for some advice to improve.
1. Can I get the light to stay on while waiting for wake-word and then it changes and flashes while waiting for the person to stop speaking?
2. Is there a way to have volume control via the voice?
3. Is there a way to interrupt the assistant during long periods of responses. I have ollama as my assistant and sometimes I dont want to hear the responses all the time.
RE: how to use AS ESP32-S3 voice assistant in home assistant - admin - 12-04-2024
point 1 and point 2 can config by esphome.
RE: how to use AS ESP32-S3 voice assistant in home assistant - biofects - 12-06-2024
I'm really loving this product and trying things with Yaml to get the features I want. I need some help if anyone can
I am adding options to always show the light and change colors and pulse and I want a way to control volume from Home assistant interface. I am close but its all not working.
I also have a bug that if I want to set colors in home assistant interface, they do not align with the colors showing on device.
the attached image you can see I have purple selected but the device shows blue
Oh 1 more thing I wanted to add a ding.wav but could not see how to add a file any help would be great there was well
This is my Yaml
Code:
esphome:
name: sarah
friendly_name: SARAH
platformio_options:
board_build.flash_mode: dio
on_boot:
- light.turn_on:
id: led_ww
blue: 100%
brightness: 60%
effect: fast pulse
esp32:
board: esp32-s3-devkitc-1
framework:
type: esp-idf
sdkconfig_options:
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
CONFIG_AUDIO_BOARD_CUSTOM: "y"
psram:
mode: octal # quad for N8R2 and octal for N16R8
speed: 80MHz
# Enable logging
logger:
hardware_uart: USB_SERIAL_JTAG
# Enable Home Assistant API
api:
encryption:
key: "************************************"
on_client_connected:
then:
- delay: 50ms
- light.turn_off: led_ww
- micro_wake_word.start:
on_client_disconnected:
then:
- voice_assistant.stop:
ota:
- platform: esphome
password: "*******************************"
wifi:
ssid: "My-SSID"
password: "***********************"
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Esp32-S3-Wake-Word"
password: "********************"
captive_portal:
button:
- platform: restart
name: "Restart"
id: but_rest
globals:
- id: current_volume
type: float
initial_value: '4.0'
number:
- platform: template
name: "Speaker Volume"
id: speaker_volume
optimistic: true
min_value: 0.5
max_value: 10.0
step: 0.5
restore_value: true
initial_value: 4.0
mode: slider
set_action:
- lambda: |-
id(va).set_volume_multiplier(x);
update_interval: 60s
switch:
- platform: template
id: mute
name: mute
optimistic: true
on_turn_on:
- micro_wake_word.stop:
- voice_assistant.stop:
- light.turn_on:
id: led_ww
red: 100%
green: 0%
blue: 0%
brightness: 60%
effect: fast pulse
- delay: 2s
- light.turn_off:
id: led_ww
- light.turn_on:
id: led_ww
red: 100%
green: 0%
blue: 0%
brightness: 30%
on_turn_off:
- micro_wake_word.start:
- light.turn_on:
id: led_ww
red: 0%
green: 100%
blue: 0%
brightness: 60%
effect: fast pulse
- delay: 2s
- light.turn_off:
id: led_ww
light:
- platform: esp32_rmt_led_strip
id: led_ww
rgb_order: GRB
pin: GPIO16
num_leds: 1
rmt_channel: 0
chipset: ws2812
name: "on board light"
effects:
- pulse:
- pulse:
name: "Fast Pulse"
transition_length: 0.5s
update_interval: 0.5s
min_brightness: 0%
max_brightness: 100%
# Audio and Voice Assistant Config
i2s_audio:
- id: i2s_in
i2s_lrclk_pin: GPIO3 #WS
i2s_bclk_pin: GPIO2 #SCK
- id: i2s_speaker
i2s_lrclk_pin: GPIO6 #LRC
i2s_bclk_pin: GPIO7 #BLCK
microphone:
- platform: i2s_audio
id: va_mic
adc_type: external
i2s_din_pin: GPIO4 #SD pin on the INMP441
channel: left
pdm: false
i2s_audio_id: i2s_in
bits_per_sample: 32 bit
speaker:
platform: i2s_audio
id: va_speaker
i2s_audio_id: i2s_speaker
dac_type: external
i2s_dout_pin: GPIO8 # DIN Pin of the MAX98357A Audio Amplifier
channel: mono
micro_wake_word:
on_wake_word_detected:
# then:
- voice_assistant.start:
wake_word: !lambda return wake_word;
- light.turn_on:
id: led_ww
red: 30%
green: 30%
blue: 70%
brightness: 60%
effect: fast pulse
models:
- model: hey_jarvis
voice_assistant:
id: va
microphone: va_mic
noise_suppression_level: 2.0
volume_multiplier: 4.0
speaker: va_speaker
on_stt_end:
then:
- light.turn_on:
id: led_ww
red: 30%
green: 30%
blue: 70%
brightness: 60%
effect: none
on_error:
- micro_wake_word.start:
on_end:
then:
- light.turn_on:
id: led_ww
red: 30%
green: 30%
blue: 70%
brightness: 60%
effect: none
- wait_until:
not:
voice_assistant.is_running:
- micro_wake_word.start: RE: how to use AS ESP32-S3 voice assistant in home assistant - admin - 12-07-2024
about the color, this line:
rgb_order: GRB
you can test with different mode:
Valid values are RGB, RBG, GRB, GBR, BRG and BGR. Defaults to RGB.
there is one mode is correct.
RE: how to use AS ESP32-S3 voice assistant in home assistant - biofects - 12-07-2024
(12-07-2024, 12:09 AM)admin Wrote: about the color, this line:
rgb_order: GRB
you can test with different mode:
Valid values are RGB, RBG, GRB, GBR, BRG and BGR. Defaults to RGB.
there is one mode is correct.
Yes, I should have started there, If I use RGB that matches the color!
Any idea how to get the volume to slider I am making to change the volume on the device? And if there is a way to upload a small .wav file to have it make a ding when enabled?
RE: how to use AS ESP32-S3 voice assistant in home assistant - admin - 12-07-2024
see here in details: https://esphome.io/components/speaker/index.html
RE: how to use AS ESP32-S3 voice assistant in home assistant - eyevisions - 12-08-2024
Also just beginning with ESP and find it hard to follow. So keep on reading
like the idea for volume control. Is there also a way to use the speaker to play media to it? I like to use bought leds so i flash it a second time and change gpio en leaf the RGB. There must be a beter way
RE: how to use AS ESP32-S3 voice assistant in home assistant - admin - 12-08-2024
sure, you can use media player component in ESPHome.
RE: how to use AS ESP32-S3 voice assistant in home assistant - biofects - 12-10-2024
(12-08-2024, 11:33 AM)eyevisions Wrote: Also just beginning with ESP and find it hard to follow. So keep on reading
I will be adding that to yaml, once i get the volume working properly its close but something is still off