Paul Kompfner
1c3e4e34e5
Minor fix to AWS Bedrock console logging to handle image messages in the context
2025-09-08 17:10:11 -04:00
Paul Kompfner
623c660027
Remove debugging comment
2025-09-08 17:01:51 -04:00
Paul Kompfner
a3e65ab3b5
The VisionImageRawFrame removal and corresponding VisionImageFrameAggregator deprecation will now happen in version 0.0.85
2025-09-08 17:01:47 -04:00
Paul Kompfner
f3a4b416df
Remove VisionImageRawFrame, which was previously being handled directly by the LLM services, and deprecate the associated VisionImageFrameAggregator.
...
Removing `VisionImageRawFrame` lets us simplify LLM services' logic, getting us closer to the idealized architecture where all they care about is handling context frames.
This change is in service of getting us closer to ready to deprecate usage of `OpenAILLMContext` and subclasses in favor of the universal `LLMContext`, at least for the traditional text-to-text LLMs.
Why remove `VisionImageRawFrame` rather than deprecate? It's "internal"—only created by `VisionImageFrameAggregator`—and never intended to be used directly by users (it would be difficult to use directly anyway).
Move the logic that was once in `VisionImageFrameAggregator` directly into the examples. Reasoning:
- If `UserImageRequester` is defined in the examples, it makes sense for `UserImageProcessor` to be too, as it’s the flip side of the same coin, so to speak
- The logic is now pretty trivial
- This kind of one-shot, history-less image-describing pipeline shouldn't be common at all; it's ok for it to live in examples rather than as a dedicated class
- In the short term, this enables us to create `LLMContext`s for services that support it and `OpenAILLMContext`s for services that don't yet (AWS)
This commit also adds missing translation from OpenAI-format image context messages to AWS format. Note that this isn't a wasted effort in the face of the upcoming migration to universal `LLMContext`—this work will be reused as it has to be implemented there too.
2025-09-08 17:00:08 -04:00
Aleix Conchillo Flaqué
d55133a44f
Merge pull request #2604 from alexyzhou/feature/livekit_video_and_bug_fix
...
Feature: Add support for livekit video stream and minor bug fixes
2025-09-08 13:51:14 -07:00
Paul Kompfner
e65385c151
Tweak the Mistral-specific context messages fixup logic to handle the (mostly academic) possibility of a "tool" message appearing at the end
2025-09-08 13:55:09 -04:00
Paul Kompfner
0bb7df7a6b
Remove stray debugging message
2025-09-08 13:38:26 -04:00
Paul Kompfner
daee1ddf3b
Apply additional fixups to context messages to meet Mistral-specific requirements
2025-09-08 11:26:58 -04:00
Aleix Conchillo Flaqué
d7794abf21
audio(filters): deprecate NoisereduceFilter
2025-09-07 20:52:17 -07:00
Mark Backman
a537382816
Add OpenAIRealtimeLLMService, AzureRealtimeLLMService ( #2596 )
...
* Add OpenAI Realtime module
* Add foundational examples for OpenAI Realtime
* Add deprecation warning to OpenAIRealtimeBetaLLMService
* Add deprecation warning to AzureRealtimeBetaLLMService
* Update Changelog
2025-09-07 09:09:57 -04:00
Alex Zhou
d6432589f6
fix: fix format and lint by ruff
2025-09-06 10:50:47 +08:00
Alex Zhou
13b73d4406
feat: Add support for pipecat video stream; fix the bug of duplicate participants when connecting; fix the bug of RTVI messages sent via livekit messages;
2025-09-06 10:41:33 +08:00
Mark Backman
2f44f88b08
LiveKitTransport: Add support to send DTMF
2025-09-05 18:23:04 -04:00
Mark Backman
25747a001b
Merge pull request #2599 from pipecat-ai/mb/fix-daily-dtmf
...
DTMF: Add support for native DTMF implementation where available
2025-09-05 15:20:05 -07:00
Mark Backman
fbe4338440
DTMF: Add support for native DTMF implementation where available
2025-09-05 18:16:56 -04:00
Paul Kompfner
dc2e1d4ad3
Fix Anthropic tool-less usage
2025-09-05 11:47:31 -04:00
Paul Kompfner
246f9f3325
Raise an error when attempting to use Gemini Multimodal Live with universal LLMContext. This is exactly the same error we already have for the other s2s models, AWS Nova Sonic and OpenAI Realtime, it was just missing from this service.
2025-09-04 16:47:08 -04:00
Paul Kompfner
6e734a37f9
Fix a bug in AWSBedrockLLMService.run_inference(); it was expecting the wrong format for the system instruction
2025-09-04 13:04:15 -04:00
Paul Kompfner
f72ca2fd7d
Remove unnecessary system_instruction argument from run_inference() methods
2025-09-04 13:04:15 -04:00
Paul Kompfner
0826d72f74
Add deprecation warning for using enable_prompt_caching_beta param
2025-09-04 13:04:15 -04:00
Paul Kompfner
dc3412b2df
Bump a deprecation to 0.0.84, as 0.0.83 just shipped
2025-09-04 13:03:06 -04:00
Paul Kompfner
b2e9fd9341
Rename Anthropic enable_prompt_caching_beta parameter to just enable_prompt_caching
2025-09-04 13:03:06 -04:00
Paul Kompfner
d6205027cf
Trivial cleanup
2025-09-04 13:03:06 -04:00
Paul Kompfner
986160c077
Fix a bug where the Anthropic adapter's merge-consecutive-messages-with-the-same-role logic was unintentionally affecting the source LLMContext's messages, resulting in more and more duplication of text with each inference
2025-09-04 13:03:06 -04:00
Paul Kompfner
b56ff86fee
Minor refactor of AnthropicLLMAdapter cache-control-marker-adding logic (without really changing its behavior)
2025-09-04 13:03:06 -04:00
Paul Kompfner
5c574eaad9
Add support for universal LLMContext to Anthropic LLM service
2025-09-04 13:03:06 -04:00
Aleix Conchillo Flaqué
b609b02614
Merge pull request #2568 from ezisezis/fix-bedrock-timeouts
...
fix timeout handling in AWSBedrockLLMService
2025-09-04 09:23:28 -07:00
Aleix Conchillo Flaqué
0d9e1190d7
Merge pull request #2583 from sassanh/main
...
fix: openai image generator now initiates URLImageRawFrame with correct order of arguments
2025-09-04 09:17:51 -07:00
Ashot
daf7fed8b3
feat(asyncai): add multilingual TTS support
2025-09-04 13:58:50 +04:00
Sassan Haradji
a26647c433
fix: openai image generator now initiates URLImageRawFrame with correct order of arguments
2025-09-04 06:09:57 +03:30
Filipi Fuchter
99e237b1e2
Fixed an issue where messages received from the transport were always being resent.
2025-09-03 18:58:34 -03:00
Aleix Conchillo Flaqué
978f644f19
Merge pull request #2578 from pipecat-ai/aleix/user-speaking-frame
...
add UserSpeakingFrame and UserStartedSpeakingFrame/UserStopeedSpeakingFrame updates
2025-09-03 14:55:45 -07:00
Aleix Conchillo Flaqué
5a4c6b9618
BaseInputTransport: push UserStartedSpeakingFrame/UserStoppedSpeakingFrame upstream
2025-09-03 14:32:32 -07:00
Mark Backman
c64bc5a636
Merge pull request #2576 from joyceerhl/joyce/cerebras-default
...
fix: update default Cerebras model to GPT-OSS-120B
2025-09-03 14:10:28 -07:00
Joyce Er
a001f6f193
Switch to GPT-OSS-120B
2025-09-03 14:00:27 -07:00
Aleix Conchillo Flaqué
09d6ec1098
introduce and push UserSpeakingFrame upstream/downstream
2025-09-03 13:56:01 -07:00
Joyce Er
d8ac6f2c1a
fix: update default Cerebras model to Qwen 3 32B
2025-09-03 12:23:36 -07:00
Aleix Conchillo Flaqué
b52296450c
DeepgramSTTService: remove raising CancelledError
2025-09-03 11:24:02 -07:00
Aleix Conchillo Flaqué
c71cec04d3
ParallelPipeline: wait for CancelFrame in all branches
2025-09-03 11:23:37 -07:00
Aleix Conchillo Flaqué
d19170d8b1
Merge pull request #2565 from pipecat-ai/aleix/reorganize-transports
...
transports: reorganize module
2025-09-03 08:52:49 -07:00
Mark Backman
8b95d74193
Merge pull request #2571 from pipecat-ai/mb/fix-docs-0.0.83
...
Fix docs generation before 0.0.83 release
2025-09-03 08:52:20 -07:00
Mark Backman
3c4694a8f1
Fix docs generation before 0.0.83 release
2025-09-03 11:31:14 -04:00
Paul Kompfner
9b216116f1
Remove supports_universal_context gate from OpenAI-library-based LLM services. It's no longer needed, as they all now support universal LLMContext.
2025-09-03 09:07:18 -04:00
Paul Kompfner
7cb372ebb9
Add support for universal LLMContext to Together.ai LLM service
2025-09-03 09:07:18 -04:00
Paul Kompfner
6838bc1e51
Add a few missing LLMContext type hints
2025-09-03 09:07:18 -04:00
Paul Kompfner
e04f42167e
Add support for universal LLMContext to SambaNova LLM service
2025-09-03 09:07:18 -04:00
Paul Kompfner
91a3f63e28
Add support for universal LLMContext to Qwen LLM service
2025-09-03 09:07:18 -04:00
Paul Kompfner
d9ea02595b
Add support for universal LLMContext to Ollama LLM service
2025-09-03 09:07:18 -04:00
Paul Kompfner
5bc0e49baa
Add support for universal LLMContext to Perplexity LLM service
2025-09-03 09:07:18 -04:00
Paul Kompfner
ec138b97d9
Add support for universal LLMContext to OpenRouter LLM service
2025-09-03 09:07:18 -04:00
