Aleix Conchillo Flaqué
d8b483dc1e
frames: generate protobuf pb2 file for pipecat package
2024-05-30 14:16:54 -07:00
chadbailey59
4c3d19cc8b
Function calling ( #175 )
...
* added function calling code back
* removed old llm_context file
* added integration testing for openai
* added function calling example
* added function callbacks
* added function start callback
* fixup
* fixup
* added different return type support for function calling
* intake example working
* added frame loggers
* cleanup
* fixup
* Update openai.py
* removed function call frame types
* fixup
* re-added example
* renumbered wake phrase
* fixup for autopep8
* remove unused imports
2024-05-30 12:25:39 -05:00
Aleix Conchillo Flaqué
a3ba07c7a3
Merge pull request #193 from pipecat-ai/aleix/fix-camera-out-enabled-cpu
...
transport(output): fix high CPU usage with camera_out_enabled and no …
2024-05-31 01:25:06 +08:00
Kwindla Hultman Kramer
a1579808b2
Merge pull request #189 from pipecat-ai/khk-cartesia-etc
...
Cartesia TTS
2024-05-30 10:24:45 -07:00
Aleix Conchillo Flaqué
aecb9f5816
transport(output): fix high CPU usage with camera_out_enabled and no images
2024-05-30 10:18:43 -07:00
Aleix Conchillo Flaqué
a9472f8116
vad: fix silero vad frame processor
2024-05-30 07:50:58 -07:00
Kwindla Hultman Kramer
d5f106ae19
pr fixes
2024-05-29 23:41:35 -07:00
Kwindla Hultman Kramer
920745345a
cartesia tts support
2024-05-29 23:35:35 -07:00
Aleix Conchillo Flaqué
39f41ab25e
transports(daily): expose dialin-ready and handle timeouts
2024-05-28 18:00:09 -07:00
Aleix Conchillo Flaqué
08a15e5cdd
transports(daily): expose on_app_message
2024-05-28 17:23:34 -07:00
Aleix Conchillo Flaqué
4cd4787e4d
transports(daily): added on_call_state_updated
2024-05-28 17:23:34 -07:00
Aleix Conchillo Flaqué
65afee2808
transport(daily): add start_recording, stop_recording and stop_dialout
2024-05-28 17:16:39 -07:00
Aleix Conchillo Flaqué
6d6d9bea5a
introduce PipelineParams
2024-05-28 17:14:14 -07:00
Kwindla Hultman Kramer
3685c19b2d
moving Deepgram TTS base_url from beta to prod
2024-05-28 15:59:26 -07:00
Aleix Conchillo Flaqué
650a2b4da4
Merge pull request #174 from pipecat-ai/fix-azure-llm-service
...
services(azure): fix AzureLLMService
2024-05-25 00:27:51 +08:00
Aleix Conchillo Flaqué
c45d428551
services(google): make api_key argument mandatory
2024-05-24 09:23:00 -07:00
Aleix Conchillo Flaqué
4e594aa9b0
services: BaseOpenAILLMService.create_client() now returns the client
2024-05-24 09:04:15 -07:00
Aleix Conchillo Flaqué
32f91c5f31
services(azure): fix AzureLLMService
...
Fixes #160
2024-05-23 16:51:04 -07:00
Aleix Conchillo Flaqué
88f6436aaa
fix aiohttp response text
2024-05-23 15:51:00 -07:00
Aleix Conchillo Flaqué
2a6183aba5
transports(daily): fix dialin-ready event handling
2024-05-23 14:38:37 -07:00
Aleix Conchillo Flaqué
c8732544c7
transport(daily): add support for dial-in use cases
2024-05-23 13:56:50 -07:00
Aleix Conchillo Flaqué
8d7d1a7e24
transport(daily): add dialin-ready event
2024-05-23 07:12:31 -07:00
Aleix Conchillo Flaqué
3768cb2f2c
transport(daily): add dialout support
2024-05-22 22:44:01 -07:00
Kwindla Hultman Kramer
681250f114
add new response frame types and vision support for anthropic
2024-05-22 21:12:30 -07:00
Aleix Conchillo Flaqué
674b8bb0cd
transports(daily): add transcription logging
2024-05-22 20:41:34 -07:00
Aleix Conchillo Flaqué
e130aada72
filters(WakeCheckFilter): increase timeout to 3
2024-05-22 19:41:14 -07:00
Aleix Conchillo Flaqué
76709a9a39
enclose text between brackets when logging
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
fcec0eb812
transports(base): log when user is speaking
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
e9965347b5
processors(WakeCheckFilter): log what frame we are pushing
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
5a83f75e0d
processors: fix user response processors
2024-05-22 19:05:18 -07:00
Aleix Conchillo Flaqué
34384881bc
transport(base): clear audio output buffer if interrupted
2024-05-22 16:30:43 -07:00
Aleix Conchillo Flaqué
e1169a4e82
processors(WakeCheckFilter): push error
2024-05-22 15:44:44 -07:00
Aleix Conchillo Flaqué
0ddfa3de5b
move WakeCheckFilter to processors/filters
2024-05-22 15:44:43 -07:00
Kwindla Hultman Kramer
661aa79b7c
fix user_id str field name in TranscriptionFrame
2024-05-22 15:44:43 -07:00
Aleix Conchillo Flaqué
d5644c3ab9
vad: re-add volume exponential smoothing
2024-05-22 15:26:32 -07:00
Aleix Conchillo Flaqué
269f070051
audio: no need for compute_rms
2024-05-22 14:09:24 -07:00
Aleix Conchillo Flaqué
3342c9d7c2
services(stt): use calculate_audio_volume
2024-05-22 13:05:20 -07:00
Aleix Conchillo Flaqué
b468b2f926
audio: clamp normalized volume
2024-05-22 13:04:09 -07:00
Aleix Conchillo Flaqué
af1c7d0023
interruptions: introduce pyloudnorm to compute loudness
...
https://github.com/csteinmetz1/pyloudnorm
2024-05-22 11:52:07 -07:00
Aleix Conchillo Flaqué
979739c1b7
processors: reset aggergator before pushing
2024-05-22 11:26:08 -07:00
Aleix Conchillo Flaqué
57a568986a
processors: only interrupt asssisstant
...
We were pushing interruption frames in the audio task. This was caussing the
LLMUserResponseAggregator to push the accumulated text and then casuing the LLM
to respond.
2024-05-22 10:15:35 -07:00
Aleix Conchillo Flaqué
825738440e
transports: run threads in their own ThreadPoolExecutor
2024-05-21 18:52:27 -07:00
Aleix Conchillo Flaqué
40ccf46b4b
aggregators: accumulate full responses and take interruptions into account
2024-05-20 11:40:57 -07:00
Aleix Conchillo Flaqué
e4c990c677
aggregators: expose LLM messages
2024-05-20 10:51:37 -07:00
Aleix Conchillo Flaqué
83812f2671
transports(daily): implement DailyOutputTransport.send_message
2024-05-20 10:30:59 -07:00
Kwindla Hultman Kramer
7ffb10d7f5
add to CHANGELOG.md
2024-05-19 12:44:45 -07:00
Kwindla Hultman Kramer
66377954cb
fix up openai vision and gemini implementation
2024-05-19 12:33:57 -07:00
Kwindla Hultman Kramer
e507686cef
oops, fix openai.py
2024-05-19 11:13:39 -07:00
Kwindla Hultman Kramer
cf597a2f6b
add back in debug log line in openai.py
2024-05-19 11:08:38 -07:00
Kwindla Hultman Kramer
b337e984b3
Initial commit of Google Gemini LLM service.
...
Gemini text input works. We translate from OpenAILLMContext format
on the fly in the GoogleLLMService implementation. This commit also
implements image input (vision) in both the GoogleLLMService and in
the OpenAILLMService. Image input is a hack and needs to be revisited.
OpenAI expects images to be uploaded as base64-encoded JPEGs. Google
does not require the base64 encoding. Other than for images, we use
the OpenAI format as our standard, but base64-encoding the images
and then unencoding them in the GoogleLLMService feels wasteful.
2024-05-19 10:35:20 -07:00
