LLMService: cancel function calls on interruptions by default
This commit is contained in:
@@ -60,6 +60,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Changed
|
||||
|
||||
- Function calls are now cancelled by default if there's an interruption. To
|
||||
disable this behavior you can set `cancel_on_interruption=False` when
|
||||
registering the function call. Since function calls are executed as tasks you
|
||||
can tell if a function call has been cancelled by catching the
|
||||
`asyncio.CancelledError` exception (and don't forget to raise it again!).
|
||||
|
||||
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
|
||||
The attribute reports TTFB in seconds.
|
||||
|
||||
|
||||
@@ -181,7 +181,7 @@ class LLMService(AIService):
|
||||
handler: Any,
|
||||
start_callback=None,
|
||||
*,
|
||||
cancel_on_interruption: bool = False,
|
||||
cancel_on_interruption: bool = True,
|
||||
):
|
||||
# Registering a function with the function_name set to None will run
|
||||
# that handler for all functions
|
||||
|
||||
Reference in New Issue
Block a user