30 lines
1.5 KiB
Markdown
30 lines
1.5 KiB
Markdown
- Added additional functionality related to "thinking", for Google and Anthropic
|
|
LLMs.
|
|
|
|
1. New typed parameters for Google and Anthropic LLMs that control the
|
|
models' thinking behavior (like how much thinking to do, and whether to
|
|
output thoughts or thought summaries):
|
|
- `AnthropicLLMService.ThinkingConfig`
|
|
- `GoogleLLMService.ThinkingConfig`
|
|
2. New frames for representing thoughts output by LLMs:
|
|
- `LLMThoughtStartFrame`
|
|
- `LLMThoughtTextFrame`
|
|
- `LLMThoughtEndFrame`
|
|
3. A generic mechanism for recording LLM thoughts to context, used
|
|
specifically to support Anthropic, whose thought signatures are expected to
|
|
appear alongside the text of the thoughts within assistant context
|
|
messages. See:
|
|
- `LLMThoughtEndFrame.signature`
|
|
- `LLMAssistantAggregator` handling of the above field
|
|
- `AnthropicLLMAdapter` handling of `"thought"` context messages
|
|
4. Google-specific logic for inserting thought signatures into the context,
|
|
to help maintain thinking continuity in a chain of LLM calls. See:
|
|
- `GoogleLLMService` sending `LLMMessagesAppendFrame`s to add LLM-specific
|
|
`"thought_signature"` messages to context
|
|
- `GeminiLLMAdapter` handling of `"thought_signature"` messages
|
|
5. An expansion of `TranscriptProcessor` to process LLM thoughts in addition
|
|
to user and assistant utterances. See:
|
|
- `TranscriptProcessor(process_thoughts=True)` (defaults to `False`)
|
|
- `ThoughtTranscriptionMessage`, which is now also emitted with the
|
|
`"on_transcript_update"` event
|