Merge pull request #2002 from pipecat-ai/mb/google-fix-ttfb
Fix: GoogleLLMService TTFB
This commit is contained in:
10
CHANGELOG.md
10
CHANGELOG.md
@@ -5,6 +5,16 @@ All notable changes to **Pipecat** will be documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Fixed an issue where `GoogleLLMService`'s TTFB value was incorrect.
|
||||||
|
|
||||||
|
### Other
|
||||||
|
|
||||||
|
- Rename `14e-function-calling-gemini.py` to `14e-function-calling-google.py`.
|
||||||
|
|
||||||
## [0.0.71] - 2025-06-10
|
## [0.0.71] - 2025-06-10
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
@@ -555,10 +555,11 @@ class GoogleLLMService(LLMService):
|
|||||||
contents=messages,
|
contents=messages,
|
||||||
config=generation_config,
|
config=generation_config,
|
||||||
)
|
)
|
||||||
await self.stop_ttfb_metrics()
|
|
||||||
|
|
||||||
function_calls = []
|
function_calls = []
|
||||||
async for chunk in response:
|
async for chunk in response:
|
||||||
|
# Stop TTFB metrics after the first chunk
|
||||||
|
await self.stop_ttfb_metrics()
|
||||||
if chunk.usage_metadata:
|
if chunk.usage_metadata:
|
||||||
prompt_tokens += chunk.usage_metadata.prompt_token_count or 0
|
prompt_tokens += chunk.usage_metadata.prompt_token_count or 0
|
||||||
completion_tokens += chunk.usage_metadata.candidates_token_count or 0
|
completion_tokens += chunk.usage_metadata.candidates_token_count or 0
|
||||||
|
|||||||
Reference in New Issue
Block a user