marcus-daily
a2e76bcad8
Smart Turn V3 support
2025-09-11 16:04:56 +01:00
Mark Backman
8e8e42717b
Add to and from phone information to the development runner
2025-09-11 10:44:21 -04:00
kompfner
b31322e38e
Merge pull request #2619 from pipecat-ai/pk/aws-universal-context
...
Expand universal `LLMContext` support to AWS Bedrock
2025-09-11 09:33:08 -04:00
Jessie Wei
305108be9a
[TwilioFrameSerializer]: Add parameter validation
2025-09-10 23:00:15 +00:00
Jessie Wei
2e1f397d17
Support TwilioFrameSerializer region/edge settings to that the call is terminated successfully for regions other than default us1 region
2025-09-10 14:30:10 +00:00
James Hush
e8f60c7c6f
Handle missing rawResponse in transcription messages ( #2623 )
...
* Handle missing rawResponse in transcription messages
- Use message.get('rawResponse', {}) to safely access rawResponse field
- Default is_final to False when rawResponse is missing
- Add proper type annotations for better code clarity
- Minor import formatting cleanup
This prevents KeyError crashes when transcription messages from Daily's API
don't include the rawResponse field in edge cases.
* docs: add changelog line
2025-09-10 15:03:23 +08:00
Paul Kompfner
8ccd220a60
Add universal LLMContext support to AWSBedrockLLMService.run_inference()
2025-09-09 16:00:32 -04:00
Paul Kompfner
fe79de8f27
When converting universal LLMContext messages to AWS Bedrock expected format, automatically update non-initial "system"-role messages to "user"-role messages, as we do in other non-OpenAI LLM services
2025-09-09 15:50:03 -04:00
Paul Kompfner
75f9914f49
Add support for universal LLMContext to AWS Bedrock LLM service
2025-09-09 15:25:04 -04:00
Paul Kompfner
1c3e4e34e5
Minor fix to AWS Bedrock console logging to handle image messages in the context
2025-09-08 17:10:11 -04:00
Paul Kompfner
623c660027
Remove debugging comment
2025-09-08 17:01:51 -04:00
Paul Kompfner
a3e65ab3b5
The VisionImageRawFrame removal and corresponding VisionImageFrameAggregator deprecation will now happen in version 0.0.85
2025-09-08 17:01:47 -04:00
Paul Kompfner
f3a4b416df
Remove VisionImageRawFrame, which was previously being handled directly by the LLM services, and deprecate the associated VisionImageFrameAggregator.
...
Removing `VisionImageRawFrame` lets us simplify LLM services' logic, getting us closer to the idealized architecture where all they care about is handling context frames.
This change is in service of getting us closer to ready to deprecate usage of `OpenAILLMContext` and subclasses in favor of the universal `LLMContext`, at least for the traditional text-to-text LLMs.
Why remove `VisionImageRawFrame` rather than deprecate? It's "internal"—only created by `VisionImageFrameAggregator`—and never intended to be used directly by users (it would be difficult to use directly anyway).
Move the logic that was once in `VisionImageFrameAggregator` directly into the examples. Reasoning:
- If `UserImageRequester` is defined in the examples, it makes sense for `UserImageProcessor` to be too, as it’s the flip side of the same coin, so to speak
- The logic is now pretty trivial
- This kind of one-shot, history-less image-describing pipeline shouldn't be common at all; it's ok for it to live in examples rather than as a dedicated class
- In the short term, this enables us to create `LLMContext`s for services that support it and `OpenAILLMContext`s for services that don't yet (AWS)
This commit also adds missing translation from OpenAI-format image context messages to AWS format. Note that this isn't a wasted effort in the face of the upcoming migration to universal `LLMContext`—this work will be reused as it has to be implemented there too.
2025-09-08 17:00:08 -04:00
Aleix Conchillo Flaqué
d55133a44f
Merge pull request #2604 from alexyzhou/feature/livekit_video_and_bug_fix
...
Feature: Add support for livekit video stream and minor bug fixes
2025-09-08 13:51:14 -07:00
Paul Kompfner
e65385c151
Tweak the Mistral-specific context messages fixup logic to handle the (mostly academic) possibility of a "tool" message appearing at the end
2025-09-08 13:55:09 -04:00
Paul Kompfner
0bb7df7a6b
Remove stray debugging message
2025-09-08 13:38:26 -04:00
Paul Kompfner
daee1ddf3b
Apply additional fixups to context messages to meet Mistral-specific requirements
2025-09-08 11:26:58 -04:00
Adithya Suresh
d1f72c1c0b
Added usage metrics
2025-09-08 14:44:25 +10:00
Aleix Conchillo Flaqué
d7794abf21
audio(filters): deprecate NoisereduceFilter
2025-09-07 20:52:17 -07:00
Mark Backman
a537382816
Add OpenAIRealtimeLLMService, AzureRealtimeLLMService ( #2596 )
...
* Add OpenAI Realtime module
* Add foundational examples for OpenAI Realtime
* Add deprecation warning to OpenAIRealtimeBetaLLMService
* Add deprecation warning to AzureRealtimeBetaLLMService
* Update Changelog
2025-09-07 09:09:57 -04:00
TheNotary
7366b1aee0
adds missing InterimTranscriptionFrame import
2025-09-06 14:40:19 -05:00
Alex Zhou
d6432589f6
fix: fix format and lint by ruff
2025-09-06 10:50:47 +08:00
Alex Zhou
13b73d4406
feat: Add support for pipecat video stream; fix the bug of duplicate participants when connecting; fix the bug of RTVI messages sent via livekit messages;
2025-09-06 10:41:33 +08:00
Mark Backman
2f44f88b08
LiveKitTransport: Add support to send DTMF
2025-09-05 18:23:04 -04:00
Mark Backman
25747a001b
Merge pull request #2599 from pipecat-ai/mb/fix-daily-dtmf
...
DTMF: Add support for native DTMF implementation where available
2025-09-05 15:20:05 -07:00
Mark Backman
fbe4338440
DTMF: Add support for native DTMF implementation where available
2025-09-05 18:16:56 -04:00
Paul Kompfner
dc2e1d4ad3
Fix Anthropic tool-less usage
2025-09-05 11:47:31 -04:00
Paul Kompfner
246f9f3325
Raise an error when attempting to use Gemini Multimodal Live with universal LLMContext. This is exactly the same error we already have for the other s2s models, AWS Nova Sonic and OpenAI Realtime, it was just missing from this service.
2025-09-04 16:47:08 -04:00
Manish Kumar
4699ee8d86
docs: add docstring for voice_cloning_key and update CHANGELOG
2025-09-04 22:45:51 +05:30
Paul Kompfner
6e734a37f9
Fix a bug in AWSBedrockLLMService.run_inference(); it was expecting the wrong format for the system instruction
2025-09-04 13:04:15 -04:00
Paul Kompfner
f72ca2fd7d
Remove unnecessary system_instruction argument from run_inference() methods
2025-09-04 13:04:15 -04:00
Paul Kompfner
0826d72f74
Add deprecation warning for using enable_prompt_caching_beta param
2025-09-04 13:04:15 -04:00
Paul Kompfner
dc3412b2df
Bump a deprecation to 0.0.84, as 0.0.83 just shipped
2025-09-04 13:03:06 -04:00
Paul Kompfner
b2e9fd9341
Rename Anthropic enable_prompt_caching_beta parameter to just enable_prompt_caching
2025-09-04 13:03:06 -04:00
Paul Kompfner
d6205027cf
Trivial cleanup
2025-09-04 13:03:06 -04:00
Paul Kompfner
986160c077
Fix a bug where the Anthropic adapter's merge-consecutive-messages-with-the-same-role logic was unintentionally affecting the source LLMContext's messages, resulting in more and more duplication of text with each inference
2025-09-04 13:03:06 -04:00
Paul Kompfner
b56ff86fee
Minor refactor of AnthropicLLMAdapter cache-control-marker-adding logic (without really changing its behavior)
2025-09-04 13:03:06 -04:00
Paul Kompfner
5c574eaad9
Add support for universal LLMContext to Anthropic LLM service
2025-09-04 13:03:06 -04:00
Aleix Conchillo Flaqué
e3597801d4
AWSNovaSonicLLMService: pre-load audio cue in the constructor
2025-09-04 09:31:39 -07:00
Aleix Conchillo Flaqué
b609b02614
Merge pull request #2568 from ezisezis/fix-bedrock-timeouts
...
fix timeout handling in AWSBedrockLLMService
2025-09-04 09:23:28 -07:00
Aleix Conchillo Flaqué
0d9e1190d7
Merge pull request #2583 from sassanh/main
...
fix: openai image generator now initiates URLImageRawFrame with correct order of arguments
2025-09-04 09:17:51 -07:00
Ashot
daf7fed8b3
feat(asyncai): add multilingual TTS support
2025-09-04 13:58:50 +04:00
Adithya Suresh
cbdbdee4c0
Initial StrandsAgentsProcessor implementation
2025-09-04 16:58:58 +10:00
Sassan Haradji
a26647c433
fix: openai image generator now initiates URLImageRawFrame with correct order of arguments
2025-09-04 06:09:57 +03:30
Filipi Fuchter
99e237b1e2
Fixed an issue where messages received from the transport were always being resent.
2025-09-03 18:58:34 -03:00
Aleix Conchillo Flaqué
978f644f19
Merge pull request #2578 from pipecat-ai/aleix/user-speaking-frame
...
add UserSpeakingFrame and UserStartedSpeakingFrame/UserStopeedSpeakingFrame updates
2025-09-03 14:55:45 -07:00
Aleix Conchillo Flaqué
5a4c6b9618
BaseInputTransport: push UserStartedSpeakingFrame/UserStoppedSpeakingFrame upstream
2025-09-03 14:32:32 -07:00
Mark Backman
c64bc5a636
Merge pull request #2576 from joyceerhl/joyce/cerebras-default
...
fix: update default Cerebras model to GPT-OSS-120B
2025-09-03 14:10:28 -07:00
Joyce Er
a001f6f193
Switch to GPT-OSS-120B
2025-09-03 14:00:27 -07:00
Aleix Conchillo Flaqué
09d6ec1098
introduce and push UserSpeakingFrame upstream/downstream
2025-09-03 13:56:01 -07:00
