Document why HTTP variant doesn't use previous_response_id
Over HTTP, previous_response_id requires store=True (30-day OpenAI-side conversation storage). The WebSocket variant avoids this via a connection-local in-memory cache that works with store=False. Add comments explaining this in both class docstrings, at the store=False parameter, and in the adapter's previous_response_id note.
This commit is contained in:
@@ -95,12 +95,11 @@ class OpenAIResponsesLLMAdapter(BaseLLMAdapter[OpenAIResponsesLLMInvocationParam
|
|||||||
# If we added support for user-provided explicit
|
# If we added support for user-provided explicit
|
||||||
# `previous_response_id` and/or `conversation_id` (overriding
|
# `previous_response_id` and/or `conversation_id` (overriding
|
||||||
# internal management), we'd need to revisit this logic, as it'd
|
# internal management), we'd need to revisit this logic, as it'd
|
||||||
# be legit to provide instructions without input items. Worth
|
# be legit to provide instructions without input items. Note that
|
||||||
# noting that OpenAI's docs suggest these parameters are primarily
|
# over HTTP, `previous_response_id` requires `store=True` (30-day
|
||||||
# for development convenience rather than performance (the model
|
# OpenAI-side storage), which is why the HTTP variant doesn't use
|
||||||
# still processes the full context), and come with the tradeoff
|
# it. The WebSocket variant avoids this via a connection-local
|
||||||
# of requiring OpenAI-side 30-day conversation storage, which may
|
# in-memory cache — see the class docstrings in llm.py.
|
||||||
# not be desirable for many users.
|
|
||||||
if not input_items:
|
if not input_items:
|
||||||
params["input"] = [{"role": "developer", "content": system_instruction}]
|
params["input"] = [{"role": "developer", "content": system_instruction}]
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -223,6 +223,10 @@ class _BaseOpenAIResponsesLLMService(LLMService):
|
|||||||
params: Dict[str, Any] = {
|
params: Dict[str, Any] = {
|
||||||
"model": self._settings.model,
|
"model": self._settings.model,
|
||||||
"stream": True,
|
"stream": True,
|
||||||
|
# store=False avoids OpenAI-side 30-day conversation storage.
|
||||||
|
# The WebSocket variant's previous_response_id optimization
|
||||||
|
# still works with store=False because it uses a connection-local
|
||||||
|
# in-memory cache. See the class docstrings for details.
|
||||||
"store": False,
|
"store": False,
|
||||||
"input": invocation_params["input"],
|
"input": invocation_params["input"],
|
||||||
}
|
}
|
||||||
@@ -337,6 +341,13 @@ class OpenAIResponsesLLMService(_BaseOpenAIResponsesLLMService):
|
|||||||
Automatically uses ``previous_response_id`` to send only incremental context when
|
Automatically uses ``previous_response_id`` to send only incremental context when
|
||||||
possible, and falls back to full context on reconnection or cache miss.
|
possible, and falls back to full context on reconnection or cache miss.
|
||||||
|
|
||||||
|
The ``previous_response_id`` optimization works with ``store=False`` (the default)
|
||||||
|
because WebSocket mode uses a connection-local in-memory cache — no conversations
|
||||||
|
are stored on OpenAI's servers. This is why the HTTP variant
|
||||||
|
(``OpenAIResponsesHttpLLMService``) does not offer this optimization by default
|
||||||
|
(or at all, yet): over HTTP, ``previous_response_id`` requires ``store=True``,
|
||||||
|
which enables OpenAI-side 30-day conversation storage.
|
||||||
|
|
||||||
This is the recommended variant for real-time / conversational use.
|
This is the recommended variant for real-time / conversational use.
|
||||||
|
|
||||||
Example::
|
Example::
|
||||||
@@ -856,6 +867,15 @@ class OpenAIResponsesHttpLLMService(_BaseOpenAIResponsesLLMService):
|
|||||||
Uses server-sent events (SSE) via the OpenAI Python SDK for streaming
|
Uses server-sent events (SSE) via the OpenAI Python SDK for streaming
|
||||||
inference. Each ``_process_context`` call opens a new HTTP connection.
|
inference. Each ``_process_context`` call opens a new HTTP connection.
|
||||||
|
|
||||||
|
Unlike the WebSocket variant, this service does not use
|
||||||
|
``previous_response_id`` for incremental context delivery by default
|
||||||
|
(or at all, yet). Over HTTP, ``previous_response_id`` requires
|
||||||
|
``store=True``, which enables OpenAI-side 30-day conversation storage
|
||||||
|
— a privacy/compliance tradeoff that many users won't want. The
|
||||||
|
WebSocket variant avoids this because its ``previous_response_id``
|
||||||
|
uses a connection-local in-memory cache that works with
|
||||||
|
``store=False`` (nothing is stored long-term).
|
||||||
|
|
||||||
Example::
|
Example::
|
||||||
|
|
||||||
llm = OpenAIResponsesHttpLLMService(
|
llm = OpenAIResponsesHttpLLMService(
|
||||||
|
|||||||
Reference in New Issue
Block a user