Document why HTTP variant doesn't use previous_response_id
Over HTTP, previous_response_id requires store=True (30-day OpenAI-side conversation storage). The WebSocket variant avoids this via a connection-local in-memory cache that works with store=False. Add comments explaining this in both class docstrings, at the store=False parameter, and in the adapter's previous_response_id note.
This commit is contained in:
@@ -95,12 +95,11 @@ class OpenAIResponsesLLMAdapter(BaseLLMAdapter[OpenAIResponsesLLMInvocationParam
|
||||
# If we added support for user-provided explicit
|
||||
# `previous_response_id` and/or `conversation_id` (overriding
|
||||
# internal management), we'd need to revisit this logic, as it'd
|
||||
# be legit to provide instructions without input items. Worth
|
||||
# noting that OpenAI's docs suggest these parameters are primarily
|
||||
# for development convenience rather than performance (the model
|
||||
# still processes the full context), and come with the tradeoff
|
||||
# of requiring OpenAI-side 30-day conversation storage, which may
|
||||
# not be desirable for many users.
|
||||
# be legit to provide instructions without input items. Note that
|
||||
# over HTTP, `previous_response_id` requires `store=True` (30-day
|
||||
# OpenAI-side storage), which is why the HTTP variant doesn't use
|
||||
# it. The WebSocket variant avoids this via a connection-local
|
||||
# in-memory cache — see the class docstrings in llm.py.
|
||||
if not input_items:
|
||||
params["input"] = [{"role": "developer", "content": system_instruction}]
|
||||
else:
|
||||
|
||||
@@ -223,6 +223,10 @@ class _BaseOpenAIResponsesLLMService(LLMService):
|
||||
params: Dict[str, Any] = {
|
||||
"model": self._settings.model,
|
||||
"stream": True,
|
||||
# store=False avoids OpenAI-side 30-day conversation storage.
|
||||
# The WebSocket variant's previous_response_id optimization
|
||||
# still works with store=False because it uses a connection-local
|
||||
# in-memory cache. See the class docstrings for details.
|
||||
"store": False,
|
||||
"input": invocation_params["input"],
|
||||
}
|
||||
@@ -337,6 +341,13 @@ class OpenAIResponsesLLMService(_BaseOpenAIResponsesLLMService):
|
||||
Automatically uses ``previous_response_id`` to send only incremental context when
|
||||
possible, and falls back to full context on reconnection or cache miss.
|
||||
|
||||
The ``previous_response_id`` optimization works with ``store=False`` (the default)
|
||||
because WebSocket mode uses a connection-local in-memory cache — no conversations
|
||||
are stored on OpenAI's servers. This is why the HTTP variant
|
||||
(``OpenAIResponsesHttpLLMService``) does not offer this optimization by default
|
||||
(or at all, yet): over HTTP, ``previous_response_id`` requires ``store=True``,
|
||||
which enables OpenAI-side 30-day conversation storage.
|
||||
|
||||
This is the recommended variant for real-time / conversational use.
|
||||
|
||||
Example::
|
||||
@@ -856,6 +867,15 @@ class OpenAIResponsesHttpLLMService(_BaseOpenAIResponsesLLMService):
|
||||
Uses server-sent events (SSE) via the OpenAI Python SDK for streaming
|
||||
inference. Each ``_process_context`` call opens a new HTTP connection.
|
||||
|
||||
Unlike the WebSocket variant, this service does not use
|
||||
``previous_response_id`` for incremental context delivery by default
|
||||
(or at all, yet). Over HTTP, ``previous_response_id`` requires
|
||||
``store=True``, which enables OpenAI-side 30-day conversation storage
|
||||
— a privacy/compliance tradeoff that many users won't want. The
|
||||
WebSocket variant avoids this because its ``previous_response_id``
|
||||
uses a connection-local in-memory cache that works with
|
||||
``store=False`` (nothing is stored long-term).
|
||||
|
||||
Example::
|
||||
|
||||
llm = OpenAIResponsesHttpLLMService(
|
||||
|
||||
Reference in New Issue
Block a user