Merge pull request #2002 from pipecat-ai/mb/google-fix-ttfb

Fix: GoogleLLMService TTFB
This commit is contained in:
Mark Backman
2025-06-12 12:10:01 -04:00
committed by GitHub
3 changed files with 12 additions and 1 deletions

View File

@@ -5,6 +5,16 @@ All notable changes to **Pipecat** will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Fixed
- Fixed an issue where `GoogleLLMService`'s TTFB value was incorrect.
### Other
- Rename `14e-function-calling-gemini.py` to `14e-function-calling-google.py`.
## [0.0.71] - 2025-06-10
### Added

View File

@@ -555,10 +555,11 @@ class GoogleLLMService(LLMService):
contents=messages,
config=generation_config,
)
await self.stop_ttfb_metrics()
function_calls = []
async for chunk in response:
# Stop TTFB metrics after the first chunk
await self.stop_ttfb_metrics()
if chunk.usage_metadata:
prompt_tokens += chunk.usage_metadata.prompt_token_count or 0
completion_tokens += chunk.usage_metadata.candidates_token_count or 0