wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update GeminiMultimodalLiveLLMService to work with the TranscriptProcessor

Browse Source
This commit is contained in:
Mark Backman
2025-03-14 14:00:38 -04:00
parent 3f002f8ffb
commit acd0660f66
2 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGELOG.md
View File

@@ -93,7 +93,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Updated `TranscriptProcessor` to support text output from - Updated `TranscriptProcessor` to support text output from
`OpenAIRealtimeBetaLLMService`. `OpenAIRealtimeBetaLLMService`.
- `OpenAIRealtimeBetaLLMService` now pushes a `TTSTextFrame`. - `OpenAIRealtimeBetaLLMService` and `GeminiMultimodalLiveLLMService` now push
a `TTSTextFrame`.
- Updated the default mode for `CartesiaTTSService` and - Updated the default mode for `CartesiaTTSService` and
`CartesiaHttpTTSService` to `sonic-2`. `CartesiaHttpTTSService` to `sonic-2`.

2
src/pipecat/services/gemini_multimodal_live/gemini.py
View File

@@ -38,6 +38,7 @@ from pipecat.frames.frames import (
TTSAudioRawFrame, TTSAudioRawFrame,
TTSStartedFrame, TTSStartedFrame,
TTSStoppedFrame, TTSStoppedFrame,
TTSTextFrame,
UserStartedSpeakingFrame, UserStartedSpeakingFrame,
UserStoppedSpeakingFrame, UserStoppedSpeakingFrame,
) )
@@ -312,6 +313,7 @@ class GeminiMultimodalLiveLLMService(LLMService):
# context.add_message({"role": "assistant", "content": [{"type": "text", "text": text}]}) # context.add_message({"role": "assistant", "content": [{"type": "text", "text": text}]})
await self.push_frame(LLMFullResponseStartFrame()) await self.push_frame(LLMFullResponseStartFrame())
await self.push_frame(LLMTextFrame(text=text)) await self.push_frame(LLMTextFrame(text=text))
await self.push_frame(TTSTextFrame(text=text))
await self.push_frame(LLMFullResponseEndFrame()) await self.push_frame(LLMFullResponseEndFrame())
async def _transcribe_audio(self, audio, context): async def _transcribe_audio(self, audio, context):