wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add WebSocket-based OpenAI Responses LLM service with previous_response_id optimization

Browse Source 
Introduce a WebSocket variant of the OpenAI Responses API service that
maintains a persistent connection to wss://api.openai.com/v1/responses
for lower-latency inference. The WebSocket variant automatically uses
previous_response_id to send only incremental context when possible,
falling back to full context on reconnection or cache miss.

The WebSocket variant becomes the new default OpenAIResponsesLLMService,
and the HTTP variant is renamed to OpenAIResponsesHttpLLMService. Both
share a private base class with common settings, parameter building,
and run_inference (always HTTP) logic.
This commit is contained in:
Paul Kompfner
2026-03-25 12:06:59 -04:00
parent d1eb2699f3
commit f2a8a9e753
11 changed files with 2297 additions and 114 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
scripts/evals/run-release-evals.py
View File

@@ -149,6 +149,7 @@ TESTS_07 = [
("07zk-interruptible-resembleai.py", EVAL_SIMPLE_MATH),
("07zl-interruptible-smallest.py", EVAL_SIMPLE_MATH),
("07-interruptible-openai-responses.py", EVAL_SIMPLE_MATH),
("07-interruptible-openai-responses-http.py", EVAL_SIMPLE_MATH),
# Needs a local XTTS docker instance running.
# ("07i-interruptible-xtts.py", EVAL_SIMPLE_MATH),
]
@@ -156,6 +157,7 @@ TESTS_07 = [
TESTS_12 = [
("12-describe-image-openai.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
("12-describe-image-openai-responses.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
("12-describe-image-openai-responses-http.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
("12a-describe-image-anthropic.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
("12b-describe-image-aws.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
("12c-describe-image-gemini-flash.py", EVAL_VISION_IMAGE(eval_speaks_first=True)),
@@ -192,6 +194,10 @@ TESTS_14 = [
("14w-function-calling-mistral.py", EVAL_WEATHER),
("14y-function-calling-sarvam.py", EVAL_WEATHER),
("14z-function-calling-novita.py", EVAL_WEATHER),
("14-function-calling-openai-responses.py", EVAL_WEATHER),
("14-function-calling-openai-responses.py", EVAL_WEATHER_AND_RESTAURANT),
("14-function-calling-openai-responses-http.py", EVAL_WEATHER),
("14-function-calling-openai-responses-http.py", EVAL_WEATHER_AND_RESTAURANT),
# Video
("14d-function-calling-anthropic-video.py", EVAL_VISION_CAMERA),
("14d-function-calling-aws-video.py", EVAL_VISION_CAMERA),
@@ -199,6 +205,7 @@ TESTS_14 = [
("14d-function-calling-moondream-video.py", EVAL_VISION_CAMERA),
("14d-function-calling-openai-video.py", EVAL_VISION_CAMERA),
("14d-function-calling-openai-responses-video.py", EVAL_VISION_CAMERA),
("14d-function-calling-openai-responses-video-http.py", EVAL_VISION_CAMERA),
# Currently not working.
# ("14c-function-calling-together.py", EVAL_WEATHER),
# ("14l-function-calling-deepseek.py", EVAL_WEATHER),