LLMService: cancel function calls on interruptions by default

This commit is contained in:
Aleix Conchillo Flaqué
2025-04-28 14:12:26 -07:00
parent 1eb50ad88f
commit 4809684a13
2 changed files with 7 additions and 1 deletions

View File

@@ -60,6 +60,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- Function calls are now cancelled by default if there's an interruption. To
disable this behavior you can set `cancel_on_interruption=False` when
registering the function call. Since function calls are executed as tasks you
can tell if a function call has been cancelled by catching the
`asyncio.CancelledError` exception (and don't forget to raise it again!).
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
The attribute reports TTFB in seconds.

View File

@@ -181,7 +181,7 @@ class LLMService(AIService):
handler: Any,
start_callback=None,
*,
cancel_on_interruption: bool = False,
cancel_on_interruption: bool = True,
):
# Registering a function with the function_name set to None will run
# that handler for all functions