diff --git a/CHANGELOG.md b/CHANGELOG.md index 359b2a2f4..a152022f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,85 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- Expanded support for univeral `LLMContext` to `OpenAIRealtimeLLMService`. + As a reminder, the context-setup pattern when using `LLMContext` is: + + ```python + context = LLMContext(messages, tools) + context_aggregator = LLMContextAggregatorPair(context) + ``` + + (Note that even though `OpenAIRealtimeLLMService` now supports the universal + `LLMContext`, it is not meant to be swapped out for another LLM service at + runtime.) + + Note: `TranscriptionFrame`s now go upstream from `OpenAIRealtimeLLMService`, + so if you're using `TranscriptProcessor`, say, you'll want to adjust + accordingly: + + ```python + pipeline = Pipeline( + [ + transport.input(), + context_aggregator.user(), + + # BEFORE + llm, + transcript.user(), + + # AFTER + transcript.user(), + llm, + + transport.output(), + transcript.assistant(), + context_aggregator.assistant(), + ] + ) + ``` + + Also worth noting: whether or not you use the new context-setup pattern with + `OpenAIRealtimeLLMService`, some types have changed under the hood: + + ```python + ## BEFORE: + + # Context aggregator type + context_aggregator: OpenAIContextAggregatorPair + + # Context frame type + frame: OpenAILLMContextFrame + + # Context type + context: OpenAIRealtimeLLMContext + # or + context: OpenAILLMContext + + # Reading messages from context + messages = context.messages + + ## AFTER: + + # Context aggregator type + context_aggregator: LLMContextAggregatorPair + + # Context frame type + frame: LLMContextFrame + + # Context type + context: LLMContext + + # Reading messages from context + messages = context.get_messages() + ``` + + Also note that `RealtimeMessagesUpdateFrame` and + `RealtimeFunctionCallResultFrame` have been deprecated, since they're no + longer used by `OpenAIRealtimeLLMService`. OpenAI Realtime now works more + like other LLM services in Pipecat, relying on updates to its context, pushed + by context aggregators, to update its internal state. Listen for + `LLMContextFrame`s for context updates. + - Expanded support for universal `LLMContext` to `GeminiLiveLLMService`. As a reminder, the context-setup pattern when using `LLMContext` is: @@ -79,6 +158,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Deprecated +- The `send_transcription_frames` argument to `OpenAIRealtimeLLMService` is + deprecated. Transcription frames are now always sent. They go upstream, to be + handled by the user context aggregator. See "Added" section for details. + +- Types in `pipecat.services.openai.realtime.context` and + `pipecat.services.openai.realtime.frames` are deprecated, as they're no + longer used by `OpenAIRealtimeLLMService`. See "Added" section for details. + - `SimliVideoService` `simli_config` parameter is deprecated. Use `api_key` and `face_id` parameters instead. @@ -200,8 +287,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 deprecated. Transcription frames are now always sent. They go upstream, to be handled by the user context aggregator. See "Added" section for details. -- Types in `pipecat.services.aws.nova_sonic.context` have been deprecated due - to changes to support `LLMContext`. See "Changed" section for details. +- Types in `pipecat.services.aws.nova_sonic.context` are deprecated, as they're + no longer used by `AWSNovaSonicLLMService`. See "Added" section for + details. ### Fixed @@ -5375,4 +5463,4 @@ a bit. ## [0.0.2] - 2024-03-12 -Initial public release. \ No newline at end of file +Initial public release.