Mark Backman
153201542b
Fix foundational 30 example to output TTSTextFrames synced to audio
2025-11-18 13:29:06 -05:00
Aleix Conchillo Flaqué
9f45ad4d2e
LLMContext: create_image_message/create_audio_message are now async
2025-11-18 09:04:40 -08:00
Paul Kompfner
5095fc6a64
Update Moondream example so that Moondream service output makes it into the context, even if the TTS service is disabled
2025-11-17 15:16:19 -05:00
Filipi Fuchter
04dbbabc03
Introduced a minimum confidence parameter in DeepgramFluxSTTService to avoid generating transcriptions below a defined threshold.
2025-11-17 09:54:30 -03:00
Mark Backman
74a0e8c88d
Merge pull request #3050 from ai-coustics/aic-vad-analyzer
...
feat(ai-coustics): add ai-coustics integrated VAD
2025-11-14 08:11:15 -05:00
kompfner
e83ac82bf3
Merge pull request #3042 from pipecat-ai/pk/follow-up-inter-frame-spaces
...
Follow-up to #3041
2025-11-13 11:03:06 -05:00
Mark Backman
edbf96b3c5
Update GeminiTTSService for streaming, other Google TTS improvements
2025-11-13 10:22:34 -05:00
Paul Kompfner
8851d18f92
Tweak the LLM prompt again to try to fix the issue of LLMs sometimes omitting punctuation in their output.
2025-11-13 10:02:33 -05:00
Mark Backman
0e37658f8d
Add ElevenLabsRealtimeSTTService
2025-11-13 09:49:05 -05:00
Corvin Jaedicke
a7b2052b38
add ai-coustics VAD
2025-11-13 14:20:35 +01:00
Paul Kompfner
1802f949ef
Fix an issue with some examples where punctuation was missing from the LLM output, by tweaking the LLM prompt.
2025-11-12 17:12:03 -05:00
Paul Kompfner
5222ff99de
Apply includes_inter_frame_spaces = True in all LLM and TTS services that need it.
...
Note that for `LLMTextFrame`s, the right behavior is pretty much always `includes_inter_frame_spaces = True`. I decided *not* to go ahead and make that the default for `LLMTextFrame`s, though, simply to not introduce a subtle behavior change for creative/unexpected use-cases that were relying on text in hand-crafted `LLMTextFrame`s being handled a certain way. Ditto for `TTSTextFrame`s.
Also, fix an issue in `NeuphonicTTSService` where it wasn't pushing `TTSTextFrame`s.
Also, fix the broken `SarvamHttpTTSService` example.
Also, add a couple of missing examples.
2025-11-12 15:10:11 -05:00
Aleix Conchillo Flaqué
0ed430e7e2
examples(foundational): use DeepgramSTTService in 07
2025-11-07 11:34:11 -08:00
Aleix Conchillo Flaqué
4f1468e0fa
scripts(evals): improve eval prompt
2025-11-07 10:05:46 -08:00
Paul Kompfner
359d220162
Document a OpenAIRealtimeLLMService gotcha in an example.
2025-11-07 10:32:27 -05:00
Paul Kompfner
c3306bb4f2
Support for passing in a ToolsSchema in lieu of a list of provider-specific dicts when updating OpenAIRealtimeLLMService using LLMUpdateSettingsFrame.
2025-11-07 10:18:29 -05:00
Mark Backman
1fb6d6bd23
GoogleSTTService: Add more robust handling of 409 errors
2025-11-06 14:35:53 -05:00
Mark Backman
9f2ddcc5f4
Merge pull request #2927 from pipecat-ai/marcus/2025-10-28_sample_rtvi_fix
...
Add RTVIProcessor to foundational example 38b
2025-11-06 10:19:10 -05:00
Mark Backman
961e28517e
Remove arg from RTVIProcessor
2025-11-06 10:16:31 -05:00
Paul Kompfner
13d6078ea0
Minor tweak to an example for clarity.
2025-11-05 15:30:01 -05:00
Paul Kompfner
9ce33f23b9
Add an example demonstrating MCP usage with a speech-to-speech service (GeminiLiveLLMService) using the pattern of passing in tools in the constructor
2025-11-05 15:29:04 -05:00
Paul Kompfner
bee4165ba4
Add LLMSwitcher.register_direct_function()
2025-11-05 15:28:19 -05:00
Paul Kompfner
0184493711
Update the service switcher example to illustrate registering tools on all LLMs in a switcher
2025-11-05 15:27:00 -05:00
vipyne
b7a4d7371c
wrap tools = await mcp.register_tools(llm) in try in examples
2025-11-04 09:01:12 -06:00
vipyne
ef88d6a2ea
update example 39-mcp-stdio.py to use different mcp server
...
https://www.loom.com/share/a9f0a270261d4c6cb054ab2b4dcd6084
SO to Rijksmuseum MCP
https://github.com/r-huijts/rijksmuseum-mcp
2025-11-04 09:01:12 -06:00
Mark Backman
0abc699f24
Merge pull request #2964 from pipecat-ai/mb/14j-nim-updates
...
Fix 14j foundational example
2025-11-04 07:24:53 -05:00
Mark Backman
1c53a5fd01
Fix 14j foundational example
2025-11-03 14:57:44 -05:00
Paul Kompfner
87131850bc
GeminiLiveLLMService supports context-provided system instruction and tools
2025-11-03 10:30:46 -05:00
shreyas-sarvam
d680ec2e69
Merge branch 'main' into sarvam/stt
2025-10-31 23:09:47 +05:30
Mark Backman
7eb880c5e8
Add DeepgramHttpTTSService
2025-10-31 11:39:32 -04:00
Aleix Conchillo Flaqué
4fa0de6660
Merge pull request #2947 from pipecat-ai/aleix/rename-add-to-context
...
UserImageRawFrame: rename add_to_context to append_to_context
2025-10-31 08:29:49 -07:00
shreyas-sarvam
2d03e51109
fix: Remove unused imports, use sample_rate from base class
2025-10-31 17:31:59 +05:30
shreyas-sarvam
09a7e08cbf
Merge branch 'main' into sarvam/stt
2025-10-31 15:21:09 +05:30
shreyas-sarvam
1433df4de2
fix: Fix language param and include suggested way of handling STT response
2025-10-31 13:23:08 +05:30
Aleix Conchillo Flaqué
685d440206
UserImageRawFrame: rename add_to_context to append_to_context
2025-10-30 15:18:27 -07:00
Paul Kompfner
ac5734d0ed
Deprecate expect_stripped_words option from LLMAssistantAggregatorParams, when used with the newer LLMAssistantAggregator, which now handles word spacing automatically.
...
This commit does not change how it works in the older `LLMAssistantContextAggregator`.
2025-10-30 17:22:47 -04:00
Aleix Conchillo Flaqué
42f0490414
examples(foundational): 14-* show how to tell the LLM we are capturing an image
2025-10-30 14:02:17 -07:00
Aleix Conchillo Flaqué
19f046a338
examples(foundational): add 12d-describe-image-moondream
2025-10-30 14:02:17 -07:00
Aleix Conchillo Flaqué
ec95618b94
don't tie UserImageRawFrame with function calls
2025-10-30 14:02:17 -07:00
Aleix Conchillo Flaqué
8fa6cbac51
examples(foundational): added 14d docstrings
2025-10-30 13:08:15 -07:00
Aleix Conchillo Flaqué
3b3a215155
examples(foundational): re-add 12-* but load image from file
2025-10-30 13:08:15 -07:00
Aleix Conchillo Flaqué
d7d409df60
examples(foundational): move 12-* to 14-*-video
2025-10-30 13:08:15 -07:00
Filipi Fuchter
52b33e5106
New event handlers for the DeepgramFluxSTTService.
2025-10-30 16:09:07 -03:00
Mark Backman
222c362fa4
Merge pull request #2937 from aaronng91/speechmatics-tts
...
Add Speechmatics TTS
2025-10-30 12:30:27 -04:00
Aaron Ng
9d509bb409
address changes
2025-10-30 16:25:10 +00:00
shreyas-sarvam
8d0e7e5e16
chore: Add changelog entry, update foundational examples
2025-10-30 19:22:14 +05:30
Paul Kompfner
8f15980c67
Get rid of unnecessary new task in example file
2025-10-29 16:23:50 -04:00
Paul Kompfner
89e9acf0e1
CHANGELOG and code comment tweaks
2025-10-29 16:21:04 -04:00
Paul Kompfner
d0f52feba3
OpenAI Realtime needs the assistant context aggregator to have expect_stripped_words=False
2025-10-29 16:15:16 -04:00
Paul Kompfner
1f96cdf970
Update OpenAIRealtimeLLMService to work with LLMContext and LLMContextAggregatorPair (cont'd).
...
Make `LLMUserAggregator` push the `LLMSetToolsFrame`s, in case a speech-to-speech service that needs to handle the frame itself—like `OpenAIRealtimeLLMService`—is downstream. As far as I can tell, pushing `LLMSetToolsFrame` should otherwise have no unwanted side effects.
2025-10-29 15:43:51 -04:00
