Merge pull request #860 from pipecat-ai/mb/transcription

Add a TranscriptProcessor and new frames
This commit is contained in:
Mark Backman
2024-12-19 08:15:53 -05:00
committed by GitHub
10 changed files with 892 additions and 5 deletions

View File

@@ -21,7 +21,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`audioop-lts` (https://github.com/AbstractUmbra/audioop) to provide the same
functionality.
- Added support for more languages to ElevenLabs (Arabic, Croatian, Filipino,
- Added timestamped conversation transcript support:
- New `TranscriptProcessor` factory provides access to user and assistant
transcript processors.
- `UserTranscriptProcessor` processes user speech with timestamps from
transcription.
- `AssistantTranscriptProcessor` processes assistant responses with LLM
context timestamps.
- Messages emitted with ISO 8601 timestamps indicating when they were spoken.
- Supports all LLM formats (OpenAI, Anthropic, Google) via standard message
format.
- New examples: `28a-transcription-processor-openai.py`,
`28b-transcription-processor-anthropic.py`, and
`28c-transcription-processor-gemini.py`.
- Add support for more languages to ElevenLabs (Arabic, Croatian, Filipino,
Tamil) and PlayHT (Afrikans, Albanian, Amharic, Arabic, Bengali, Croatian,
Galician, Hebrew, Mandarin, Serbian, Tagalog, Urdu, Xhosa).