Paul Kompfner f2a8a9e753 Add WebSocket-based OpenAI Responses LLM service with previous_response_id optimization ...

Introduce a WebSocket variant of the OpenAI Responses API service that
maintains a persistent connection to wss://api.openai.com/v1/responses
for lower-latency inference. The WebSocket variant automatically uses
previous_response_id to send only incremental context when possible,
falling back to full context on reconnection or cache miss.

The WebSocket variant becomes the new default OpenAIResponsesLLMService,
and the HTTP variant is renamed to OpenAIResponsesHttpLLMService. Both
share a private base class with common settings, parameter building,
and run_inference (always HTTP) logic.

2026-03-30 09:58:56 -04:00

5.9 KiB

Raw Blame History

View Raw