Aleix Conchillo Flaqué
88f6436aaa
fix aiohttp response text
2024-05-23 15:51:00 -07:00
Aleix Conchillo Flaqué
2a6183aba5
transports(daily): fix dialin-ready event handling
2024-05-23 14:38:37 -07:00
Aleix Conchillo Flaqué
c8732544c7
transport(daily): add support for dial-in use cases
2024-05-23 13:56:50 -07:00
Aleix Conchillo Flaqué
8d7d1a7e24
transport(daily): add dialin-ready event
2024-05-23 07:12:31 -07:00
Aleix Conchillo Flaqué
3768cb2f2c
transport(daily): add dialout support
2024-05-22 22:44:01 -07:00
Kwindla Hultman Kramer
681250f114
add new response frame types and vision support for anthropic
2024-05-22 21:12:30 -07:00
Aleix Conchillo Flaqué
674b8bb0cd
transports(daily): add transcription logging
2024-05-22 20:41:34 -07:00
Aleix Conchillo Flaqué
e130aada72
filters(WakeCheckFilter): increase timeout to 3
2024-05-22 19:41:14 -07:00
Aleix Conchillo Flaqué
76709a9a39
enclose text between brackets when logging
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
fcec0eb812
transports(base): log when user is speaking
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
e9965347b5
processors(WakeCheckFilter): log what frame we are pushing
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
5a83f75e0d
processors: fix user response processors
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
34384881bc
transport(base): clear audio output buffer if interrupted
2024-05-22 16:30:43 -07:00
Aleix Conchillo Flaqué
e1169a4e82
processors(WakeCheckFilter): push error
2024-05-22 15:44:44 -07:00
Aleix Conchillo Flaqué
0ddfa3de5b
move WakeCheckFilter to processors/filters
2024-05-22 15:44:43 -07:00
Kwindla Hultman Kramer
661aa79b7c
fix user_id str field name in TranscriptionFrame
2024-05-22 15:44:43 -07:00
Aleix Conchillo Flaqué
d5644c3ab9
vad: re-add volume exponential smoothing
2024-05-22 15:26:32 -07:00
Aleix Conchillo Flaqué
269f070051
audio: no need for compute_rms
2024-05-22 14:09:24 -07:00
Aleix Conchillo Flaqué
3342c9d7c2
services(stt): use calculate_audio_volume
2024-05-22 13:05:20 -07:00
Aleix Conchillo Flaqué
b468b2f926
audio: clamp normalized volume
2024-05-22 13:04:09 -07:00
Aleix Conchillo Flaqué
af1c7d0023
interruptions: introduce pyloudnorm to compute loudness
...
https://github.com/csteinmetz1/pyloudnorm
2024-05-22 11:52:07 -07:00
Aleix Conchillo Flaqué
979739c1b7
processors: reset aggergator before pushing
2024-05-22 11:26:08 -07:00
Aleix Conchillo Flaqué
57a568986a
processors: only interrupt asssisstant
...
We were pushing interruption frames in the audio task. This was caussing the
LLMUserResponseAggregator to push the accumulated text and then casuing the LLM
to respond.
2024-05-22 10:15:35 -07:00
Aleix Conchillo Flaqué
825738440e
transports: run threads in their own ThreadPoolExecutor
2024-05-21 18:52:27 -07:00
Aleix Conchillo Flaqué
40ccf46b4b
aggregators: accumulate full responses and take interruptions into account
2024-05-20 11:40:57 -07:00
Aleix Conchillo Flaqué
e4c990c677
aggregators: expose LLM messages
2024-05-20 10:51:37 -07:00
Aleix Conchillo Flaqué
83812f2671
transports(daily): implement DailyOutputTransport.send_message
2024-05-20 10:30:59 -07:00
Kwindla Hultman Kramer
7ffb10d7f5
add to CHANGELOG.md
2024-05-19 12:44:45 -07:00
Kwindla Hultman Kramer
66377954cb
fix up openai vision and gemini implementation
2024-05-19 12:33:57 -07:00
Kwindla Hultman Kramer
e507686cef
oops, fix openai.py
2024-05-19 11:13:39 -07:00
Kwindla Hultman Kramer
cf597a2f6b
add back in debug log line in openai.py
2024-05-19 11:08:38 -07:00
Kwindla Hultman Kramer
b337e984b3
Initial commit of Google Gemini LLM service.
...
Gemini text input works. We translate from OpenAILLMContext format
on the fly in the GoogleLLMService implementation. This commit also
implements image input (vision) in both the GoogleLLMService and in
the OpenAILLMService. Image input is a hack and needs to be revisited.
OpenAI expects images to be uploaded as base64-encoded JPEGs. Google
does not require the base64 encoding. Other than for images, we use
the OpenAI format as our standard, but base64-encoding the images
and then unencoding them in the GoogleLLMService feels wasteful.
2024-05-19 10:35:20 -07:00
Aleix Conchillo Flaqué
c3bfcbd562
aggregators: clear accumulated responses if interruption happens
2024-05-19 10:21:45 -07:00
Aleix Conchillo Flaqué
810dc30d3d
examples: fix examples to use LLMFullResponseEndFrame
2024-05-19 09:39:34 -07:00
Aleix Conchillo Flaqué
435fffe1b0
add LLMFullResponseStartFrame/LLMFullResponseEndFrame
2024-05-18 09:49:38 -07:00
Aleix Conchillo Flaqué
2b8f1c4cda
services(openai): send LLMResponseStartFrame for each completion
2024-05-17 17:47:33 -07:00
Aleix Conchillo Flaqué
0e8c7a9b28
transports(output): create an downstream push frame task
2024-05-17 17:47:24 -07:00
Aleix Conchillo Flaqué
3e13678f23
vad: use exponential smoothed volume to improve speech detection
2024-05-17 17:13:31 -07:00
Aleix Conchillo Flaqué
455ec4f1fd
services(tts): always send received TextFrame downstream
2024-05-17 17:11:11 -07:00
Aleix Conchillo Flaqué
de65028061
vad: reduce default confidence back to 0.5
2024-05-17 14:39:40 -07:00
Aleix Conchillo Flaqué
34762bf604
transports: allows update allow_interruptinos when receiving StartFrame
2024-05-17 14:15:37 -07:00
Aleix Conchillo Flaqué
57121338b1
pipeline(task): cleanup processors only if we need to
2024-05-17 13:53:33 -07:00
Aleix Conchillo Flaqué
a5d246ec0c
vad: use exponential smoothing to avoid sudden changes
2024-05-17 13:53:33 -07:00
Aleix Conchillo Flaqué
f2cefeeedc
utils: move exp_smoothing to utils module
2024-05-17 13:52:18 -07:00
Aleix Conchillo Flaqué
537e72a05f
vad: introduce VADParams so you can tweak things
2024-05-17 13:52:18 -07:00
Aleix Conchillo Flaqué
efa5a061d7
silero: simplify int16 -> float32 conversion
2024-05-17 13:51:06 -07:00
Aleix Conchillo Flaqué
0bef44c2ff
introduce StartInterruptionFrame and StopInterruptionFrame
2024-05-17 13:51:06 -07:00
Aleix Conchillo Flaqué
f62fe059b1
fix issues with Ctrl-C tasks cancellation
2024-05-17 13:51:04 -07:00
Aleix Conchillo Flaqué
f432e2b17e
transports: allow adding a vad analyzer to BaseInputTransport
2024-05-17 13:50:48 -07:00
Aleix Conchillo Flaqué
dc9377fb92
add missing queue task_done()
2024-05-17 13:50:48 -07:00
