Paul Kompfner
e604e9b490
Support conversations with Gemini 3 Pro Image (model "gemini-3-pro-image-preview").
...
Prior to this change, after the model generated an image the conversation would not be able to progress. It would stall out because we were never storing the image in context, so the model would never realize it already did the work of generating an image. We didn't run into issues with Gemini 2.5 Flash Image, because that model always followed up an image with a text message.
2025-12-12 18:20:17 -05:00
Paul Kompfner
abc2ad8cbc
Avoid printing out entire thought signatures in logs
2025-12-12 13:01:45 -05:00
Paul Kompfner
64471d65f8
Clean up logic related to applying Gemini thought signatures to context messages
2025-12-12 12:53:11 -05:00
Filipi da Silva Fuchter
22288648e6
Merge pull request #3210 from pipecat-ai/filipi/heygen_liveavatar
...
Adding support for the HeyGen LiveAvatar API
2025-12-12 09:19:58 -03:00
Filipi Fuchter
a6ee040d82
Adding the changelog mentioning the HeyGen changes.
2025-12-12 08:58:48 -03:00
Filipi Fuchter
87fc860cd5
Changing the HeyGenVideoService example to use the live avatar API.
2025-12-12 08:52:10 -03:00
Filipi Fuchter
b25ad21941
Refactoring HeyGenVideoService and HeyGenTransport to work with both APIs.
2025-12-12 08:51:35 -03:00
Filipi Fuchter
debcea3baa
Adding the new HEYGEN_LIVE_AVATAR_API_KEY to the requested environment's variables.
2025-12-12 08:51:01 -03:00
Filipi Fuchter
c2abe42a64
Adding support for the HeyGen LiveAvatar API.
2025-12-12 08:49:52 -03:00
Filipi Fuchter
56dee06a29
Refactored the Interactive Avatar API to extend the HeyGen base API.
2025-12-12 08:49:16 -03:00
Filipi Fuchter
60cc14cafd
Created HeyGen base API to support both Interactive Avatar and LiveAvatar.
2025-12-12 08:48:39 -03:00
kompfner
1e98094394
Merge pull request #3175 from pipecat-ai/pk/thinking-exploration
...
Additional functionality related to thinking, for Google and Anthropic LLMs.
2025-12-11 17:15:37 -05:00
Paul Kompfner
ccdd6cde52
Fix a couple of typos in comments
2025-12-11 17:05:09 -05:00
Paul Kompfner
12979293ad
Add thinking examples to eval suite
2025-12-11 15:58:48 -05:00
Paul Kompfner
28248e9b00
Split up thinking examples so that there isn't an llm command-line arg for controlling which LLM to use. This change is preparation for adding these examples to our suite of evals.
2025-12-11 15:07:35 -05:00
Paul Kompfner
0e88ad672e
Add ThoughtTranscriptionMessage.role, which is always "assistant"
2025-12-11 14:41:16 -05:00
kompfner
f41c3dcbc3
Merge pull request #3212 from pipecat-ai/pk/nova-2-sonic
...
Nova 2 Sonic support
2025-12-11 09:36:50 -05:00
Mark Backman
645e1802f8
Merge pull request #3219 from pipecat-ai/mb/deprecate-fal-smart-turn
2025-12-10 13:13:44 -05:00
Mark Backman
6636da682c
Merge pull request #3085 from rimelabs/feature/rimeNonJsonTTsservice
...
Add RimeNonJsonTTSService for non-JSON WebSocket API support
2025-12-10 10:38:39 -05:00
Mark Backman
10a32c943f
deprecate: FalSmartTurnAnalyzer and LocalSmartTurnAnalyzer
2025-12-10 08:14:28 -05:00
Gokul Js
455579ffcc
Refactor RimeNonJsonTTSService to extend InterruptibleTTSService, removing dependency on WebsocketTTSService and streamlining audio interruption handling.
2025-12-10 04:56:52 +05:30
Paul Kompfner
c37da6ab78
In the AWS Nova Sonic example, shorten the simulated weather function call delay
2025-12-09 16:53:18 -05:00
Paul Kompfner
1892854516
In the AWS Nova Sonic example, send back "location" from the weather-fetching function to help the model associate a tool response with a tool call...if you interrupt the model while more than one function call is outbound, it seemingly can get confused about which tool result goes which call.
2025-12-09 16:27:23 -05:00
Mark Backman
735e597bf2
Merge pull request #3209 from pipecat-ai/hush/07n-prompt
...
Update system prompt in Gemini example to be more instructive
2025-12-09 15:45:46 -05:00
Vanessa Pyne
52980a69c5
Merge pull request #3215 from pipecat-ai/vp-user-bot-latency-observer-internal-var-change
...
user-bot-latency log observer internal var change
2025-12-09 13:03:29 -06:00
vipyne
ff2f1dac82
user-bot-latency log observer internal var change
2025-12-09 12:34:38 -06:00
Paul Kompfner
3cbfbb997e
Added CHANGELOG for AWS Nova 2 Sonic-related changes
2025-12-09 12:57:19 -05:00
Paul Kompfner
3e66cb50e0
Update AWS Nova Sonic example to showcase async tool calling
2025-12-09 12:44:21 -05:00
Paul Kompfner
b821dd2507
Fix a bug in AWSNovaSonicLLMService where we would mishandle cancelled tool calls in context
2025-12-09 12:12:55 -05:00
Paul Kompfner
0c5bccd1f1
Changes related to Nova 2 Sonic's support for the model speaking first
2025-12-09 11:55:23 -05:00
Paul Kompfner
926514ca18
Add support to AWSNovaSonicLLMService for new "endpointingSensitivity" parameter.
2025-12-09 11:26:43 -05:00
Paul Kompfner
ca5e668f4a
Update AWSNovaSonicLLMService docstring with more (and more up-to-date) info
2025-12-09 10:14:27 -05:00
Paul Kompfner
53de6c0b9a
Update list of supported regions in 40-aws-nova-sonic.py
2025-12-09 09:46:53 -05:00
Paul Kompfner
b22ac8292f
Update default model in AWSNovaSonicLLMService to "amazon.nova-2-sonic-v1:0"
2025-12-09 09:38:47 -05:00
James Hush
83877ab1e6
Update system prompt in Gemini example to be more instructive
...
Changed the on_client_connected system message from a direct greeting to
an instruction that tells the AI to introduce itself, giving the LLM more
flexibility in how it starts the conversation.
2025-12-09 09:04:10 +01:00
Gokul Js
2a6a0d83db
Update docstring in RimeNonJsonTTSService to clarify the focus on the current plain text protocol and note potential future support for JSON WebSocket.
2025-12-09 02:49:37 +05:30
Gokul Js
6ca117a3c1
Remove unused import of 'language' in tts.py to clean up the code and improve readability.
2025-12-09 02:45:17 +05:30
Gokul Js
4fcb099fd7
Add RimeNonJsonTTSService to support non-JSON streaming mode, enabling WebSocket streaming for the Arcana model.
2025-12-09 02:43:57 +05:30
Paul Kompfner
c5ff5cc219
Update CHANGELOG
2025-12-08 16:09:59 -05:00
Aleix Conchillo Flaqué
88289f578a
Merge pull request #3208 from pipecat-ai/thor/add-client-identification
...
add Gemini client identification
2025-12-08 13:05:04 -08:00
Paul Kompfner
229ff794d6
Better handle Gemini non-function thought signatures
2025-12-08 15:56:40 -05:00
Aleix Conchillo Flaqué
096db3eb6c
Merge pull request #3207 from pipecat-ai/aleix/voicemail-conversation-detected-event
...
VoicemailDetector: add on_conversation_detected event
2025-12-08 11:59:45 -08:00
Aleix Conchillo Flaqué
cfd1cada8c
VoicemailDetector: add on_conversation_detected event
2025-12-08 11:57:14 -08:00
Aleix Conchillo Flaqué
ee435b6f1e
update CHANGELOG
2025-12-08 11:54:09 -08:00
Aleix Conchillo Flaqué
d289b38ba7
tests(google): mock the new pipecat.version()
2025-12-08 11:51:01 -08:00
Aleix Conchillo Flaqué
b0f63c3785
pipecat: add version() function
2025-12-08 11:51:01 -08:00
Paul Kompfner
1249ee3de3
Better handle Gemini non-function thought signatures
2025-12-08 13:07:25 -05:00
Vanessa Pyne
b09d8bd595
Merge pull request #3206 from pipecat-ai/vp-update-bot-latency-observer
...
use VADUserStarted/StoppedSpeakingFrame s in user_bot_latency_log_observer.py
2025-12-08 11:37:56 -06:00
vipyne
540a48b1b6
use VADUserStarted/StoppedSpeakingFrame s in user_bot_latency_log_observer.py
2025-12-08 11:37:31 -06:00
Paul Kompfner
aa0529ff82
Update comments for accuracy
2025-12-08 11:47:06 -05:00
