Aleix Conchillo Flaqué
b094418d1e
LLMContext: add create_image_message and create_audio_message
2025-10-30 13:08:13 -07:00
Filipi da Silva Fuchter
08a1e09020
Merge pull request #2944 from pipecat-ai/filipi/flux_handlers
...
New event handlers for the DeepgramFluxSTTService.
2025-10-30 16:40:41 -03:00
Filipi Fuchter
52b33e5106
New event handlers for the DeepgramFluxSTTService.
2025-10-30 16:09:07 -03:00
Mark Backman
5db0871a20
Merge pull request #2873 from matejmarinko-soniox/main
...
Update model params for Soniox STT
2025-10-30 12:50:30 -04:00
Mark Backman
222c362fa4
Merge pull request #2937 from aaronng91/speechmatics-tts
...
Add Speechmatics TTS
2025-10-30 12:30:27 -04:00
Aaron Ng
9d509bb409
address changes
2025-10-30 16:25:10 +00:00
shreyas-sarvam
8d0e7e5e16
chore: Add changelog entry, update foundational examples
2025-10-30 19:22:14 +05:30
shreyas-sarvam
e7b8da7a83
feat: Refactor code to include language parameter, model_name and use _handle_transcription method
2025-10-30 19:01:04 +05:30
shreyas-sarvam
35c48a45cf
fix: Ruff format
2025-10-30 18:51:18 +05:30
shreyas-sarvam
14a365aa16
fix: Use message handler to handle responses
2025-10-30 17:54:32 +05:30
shreyas-sarvam
779fc0419d
Merge branch 'main' into sarvam/stt
2025-10-30 15:50:53 +05:30
Thu Nguyen
057e0c3973
Lint
2025-10-30 17:12:36 +07:00
Thu Nguyen
8a6abdd44b
feat: Add dynamic speaking_rate control for Google TTS Chirp voices
2025-10-30 17:09:41 +07:00
Mark Backman
7872fa2e88
Merge pull request #2934 from roshie548/add-cartesia-generation-config
...
feat: add generation_config support for Cartesia Sonic-3
2025-10-29 23:10:48 -04:00
Roshan
e86c546a1a
Merge branch 'main' into add-cartesia-generation-config
2025-10-29 18:31:09 -07:00
Roshan
abf34bcccf
address pr comments
2025-10-29 18:29:51 -07:00
Aleix Conchillo Flaqué
56eb633390
Merge pull request #2911 from pipecat-ai/aleix/daily-transport-improve-error-handling
...
DailyTransport: update start_dialout/start_recording return values
2025-10-29 16:28:10 -07:00
Aleix Conchillo Flaqué
6299b9db87
DailyTransport: trigger "on_error" if transcription fails to start/stop
2025-10-29 16:25:13 -07:00
Aleix Conchillo Flaqué
bcffa590a3
DailyTransport: update start_dialout/start_recording return values
2025-10-29 16:25:13 -07:00
kompfner
8b739aa444
Merge pull request #2889 from pipecat-ai/pk/openai-realtime-universal-llmcontext-2
...
Support new `LLMContext` pattern with `OpenAIRealtimeLLMService`
2025-10-29 16:54:37 -04:00
Paul Kompfner
8f15980c67
Get rid of unnecessary new task in example file
2025-10-29 16:23:50 -04:00
Paul Kompfner
89e9acf0e1
CHANGELOG and code comment tweaks
2025-10-29 16:21:04 -04:00
Paul Kompfner
ddac24e6c9
Fix a missing space in a warning message
2025-10-29 16:17:05 -04:00
Paul Kompfner
d0f52feba3
OpenAI Realtime needs the assistant context aggregator to have expect_stripped_words=False
2025-10-29 16:15:16 -04:00
Paul Kompfner
8894db4290
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Add warning about no longer pushing `TTSTextFrame`s.
2025-10-29 15:45:06 -04:00
Paul Kompfner
1f96cdf970
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Make `LLMUserAggregator` push the `LLMSetToolsFrame`s, in case a speech-to-speech service that needs to handle the frame itself—like `OpenAIRealtimeLLMService`—is downstream. As far as I can tell, pushing `LLMSetToolsFrame` should otherwise have no unwanted side effects.
2025-10-29 15:43:51 -04:00
Paul Kompfner
0282033208
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Add `LLMContext.get_messages_for_persistent_storage()` for compatibility with `OpenAILLMContext`, to avoid tripping up users who we're unknowingly migrating to using `LLMContext`.
2025-10-29 15:43:51 -04:00
Paul Kompfner
917ea27352
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Update `AzureRealtimeLLMService` example (19a) to use new `LLMContext` pattern.
2025-10-29 15:43:51 -04:00
Paul Kompfner
8c03df1463
Update some docstring arg descriptions to be a bit more current or accurate
2025-10-29 15:43:51 -04:00
Paul Kompfner
15aa76efba
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Maintain backward compatibility with functions specified in dict format.
2025-10-29 15:43:51 -04:00
Paul Kompfner
8ac421f8fd
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Remove unused imports.
2025-10-29 15:43:51 -04:00
Paul Kompfner
75b3ea9c96
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Fix tracing.
2025-10-29 15:43:51 -04:00
Paul Kompfner
95be1510ac
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Improve `OpenAIRealtimeLLMAdapter.get_messages_for_logging()`.
2025-10-29 15:43:51 -04:00
Paul Kompfner
df19011080
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Improve warning about transcription frame direction change.
2025-10-29 15:43:51 -04:00
Paul Kompfner
e42cf78e79
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Update deprecation versions.
2025-10-29 15:43:51 -04:00
Paul Kompfner
0495de52b6
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Log warning about transcription frame direction change.
2025-10-29 15:43:51 -04:00
Paul Kompfner
9bc02afd0d
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
CHANGELOG tweak.
2025-10-29 15:43:51 -04:00
Paul Kompfner
6140fdb2c9
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
In anticipation of `messages` property being added to `LLMContext` (in another PR), remove warnings about the need to use `get_messages()` instead.
2025-10-29 15:43:51 -04:00
Paul Kompfner
b6a1886dae
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
2025-10-29 15:43:51 -04:00
Paul Kompfner
42d0a097c5
Tweaks to 20b example
2025-10-29 15:43:51 -04:00
Paul Kompfner
3761804146
Make OpenAIRealtimeLLMService's websocket send method more resilient. Previously, it was possible for a websocket send attempt to occur during a disconnect.
2025-10-29 15:43:51 -04:00
Paul Kompfner
46e97c57c2
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Update 20b example to use new `LLMContext` pattern.
2025-10-29 15:43:51 -04:00
Paul Kompfner
19770b76b4
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Add back file that was removed, when it should've just been deprecated.
Also, fix version numbers in deprecation messages to match the next expected release.
2025-10-29 15:43:51 -04:00
Paul Kompfner
b34461bf93
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
2025-10-29 15:43:47 -04:00
Paul Kompfner
bab0aaf585
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Update `create_context_aggregator()` (which we're keeping around for backward compatibility) to create a `LLMContextAggregatorPair` rather than OpenAI-Realtime-specific aggregators.
2025-10-29 15:36:58 -04:00
Paul Kompfner
61944d22ef
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Implement sending tool call results to the OpenAI server based on reading context updates. This lets us use the normal assistant context aggregator and not a special OpenAI Realtime subclass that pushes up a special frame for function call results.
2025-10-29 15:36:58 -04:00
Paul Kompfner
47756319be
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Receiving a new context (via a context frame) no longer serves as a signal to reset the conversation. That’s because we’re now receiving new contexts from the user aggregator every time new messages are added, and from the assistant aggregator when function call results come in. The code pattern we're heading towards, of “diffing” each new context with the previous on, sets us up for doing more sophisticated things in the future, like sending specific messages to OpenAI to edit its internally-tracked context.
Also, remove code that was directly modifying context.
2025-10-29 15:36:58 -04:00
Paul Kompfner
5fa56df014
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Update 19b example with new pattern.
2025-10-29 15:36:58 -04:00
Paul Kompfner
8a151235c3
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Deprecate `send_transcription_frames`—transcription frames are now always sent.
2025-10-29 15:36:57 -04:00
Paul Kompfner
ec42f8c24e
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Push `TranscriptionFrame`s upstream, to be handled by the user context aggregator. This will require at least a couple of other changes:
- Updating examples to put transcript processors upstream from `OpenAIRealtimeLLMService`
- Maybe figuring out a way to preserve backward compatibility with existing pipelines that put transcript processors downstream from `OpenAIRealtimeLLMService`
- Updating `OpenAIRealtimeLLMService` to ignore new received context frames, since the upstream user context aggregator will generate those after each newly-added user message; hopefully nobody was reliant on the old behavior of resetting the conversation upon receiving a new context!
2025-10-29 15:36:57 -04:00
