Document why HTTP variant doesn't use previous_response_id

Over HTTP, previous_response_id requires store=True (30-day OpenAI-side
conversation storage). The WebSocket variant avoids this via a
connection-local in-memory cache that works with store=False. Add
comments explaining this in both class docstrings, at the store=False
parameter, and in the adapter's previous_response_id note.
This commit is contained in:
Paul Kompfner
2026-03-25 15:46:23 -04:00
parent 1c8d31de70
commit 670ce30a1c
2 changed files with 25 additions and 6 deletions

View File

@@ -95,12 +95,11 @@ class OpenAIResponsesLLMAdapter(BaseLLMAdapter[OpenAIResponsesLLMInvocationParam
# If we added support for user-provided explicit
# `previous_response_id` and/or `conversation_id` (overriding
# internal management), we'd need to revisit this logic, as it'd
# be legit to provide instructions without input items. Worth
# noting that OpenAI's docs suggest these parameters are primarily
# for development convenience rather than performance (the model
# still processes the full context), and come with the tradeoff
# of requiring OpenAI-side 30-day conversation storage, which may
# not be desirable for many users.
# be legit to provide instructions without input items. Note that
# over HTTP, `previous_response_id` requires `store=True` (30-day
# OpenAI-side storage), which is why the HTTP variant doesn't use
# it. The WebSocket variant avoids this via a connection-local
# in-memory cache — see the class docstrings in llm.py.
if not input_items:
params["input"] = [{"role": "developer", "content": system_instruction}]
else:

View File

@@ -223,6 +223,10 @@ class _BaseOpenAIResponsesLLMService(LLMService):
params: Dict[str, Any] = {
"model": self._settings.model,
"stream": True,
# store=False avoids OpenAI-side 30-day conversation storage.
# The WebSocket variant's previous_response_id optimization
# still works with store=False because it uses a connection-local
# in-memory cache. See the class docstrings for details.
"store": False,
"input": invocation_params["input"],
}
@@ -337,6 +341,13 @@ class OpenAIResponsesLLMService(_BaseOpenAIResponsesLLMService):
Automatically uses ``previous_response_id`` to send only incremental context when
possible, and falls back to full context on reconnection or cache miss.
The ``previous_response_id`` optimization works with ``store=False`` (the default)
because WebSocket mode uses a connection-local in-memory cache — no conversations
are stored on OpenAI's servers. This is why the HTTP variant
(``OpenAIResponsesHttpLLMService``) does not offer this optimization by default
(or at all, yet): over HTTP, ``previous_response_id`` requires ``store=True``,
which enables OpenAI-side 30-day conversation storage.
This is the recommended variant for real-time / conversational use.
Example::
@@ -856,6 +867,15 @@ class OpenAIResponsesHttpLLMService(_BaseOpenAIResponsesLLMService):
Uses server-sent events (SSE) via the OpenAI Python SDK for streaming
inference. Each ``_process_context`` call opens a new HTTP connection.
Unlike the WebSocket variant, this service does not use
``previous_response_id`` for incremental context delivery by default
(or at all, yet). Over HTTP, ``previous_response_id`` requires
``store=True``, which enables OpenAI-side 30-day conversation storage
— a privacy/compliance tradeoff that many users won't want. The
WebSocket variant avoids this because its ``previous_response_id``
uses a connection-local in-memory cache that works with
``store=False`` (nothing is stored long-term).
Example::
llm = OpenAIResponsesHttpLLMService(