LLMService: cancel function calls on interruptions by default
This commit is contained in:
@@ -60,6 +60,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
|
- Function calls are now cancelled by default if there's an interruption. To
|
||||||
|
disable this behavior you can set `cancel_on_interruption=False` when
|
||||||
|
registering the function call. Since function calls are executed as tasks you
|
||||||
|
can tell if a function call has been cancelled by catching the
|
||||||
|
`asyncio.CancelledError` exception (and don't forget to raise it again!).
|
||||||
|
|
||||||
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
|
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
|
||||||
The attribute reports TTFB in seconds.
|
The attribute reports TTFB in seconds.
|
||||||
|
|
||||||
|
|||||||
@@ -181,7 +181,7 @@ class LLMService(AIService):
|
|||||||
handler: Any,
|
handler: Any,
|
||||||
start_callback=None,
|
start_callback=None,
|
||||||
*,
|
*,
|
||||||
cancel_on_interruption: bool = False,
|
cancel_on_interruption: bool = True,
|
||||||
):
|
):
|
||||||
# Registering a function with the function_name set to None will run
|
# Registering a function with the function_name set to None will run
|
||||||
# that handler for all functions
|
# that handler for all functions
|
||||||
|
|||||||
Reference in New Issue
Block a user