Merge pull request #676 from RonakAgarwalVani/fix/chunk-choices-delta-none

Fix uncaught exception when accessing 'tool_calls' in NoneType delta in response handling
This commit is contained in:
Vanessa Pyne
2024-10-30 14:44:32 -05:00
committed by GitHub

View File

@@ -223,6 +223,9 @@ class BaseOpenAILLMService(LLMService):
await self.stop_ttfb_metrics()
if not chunk.choices[0].delta:
continue
if chunk.choices[0].delta.tool_calls:
# We're streaming the LLM response to enable the fastest response times.
# For text, we just yield each chunk as we receive it and count on consumers