wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,863 Commits 276 Branches 111 Tags
137bbb3d2c9ff27701e69879052d0ff62965df76
Commit Graph

6863 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yukiobata1
137bbb3d2c updated tts.py to match mark's version 2026-01-06 21:16:13 +09:00
yukiobata1
4f93d331b7 Added await to self.start_word_timestamps() 2025-12-31 19:19:21 +09:00
yukiobata1
32c6dccebe Add word-level timestamp support to Azure TTS with cumulative PTS fix 
This commit adds word boundary support to AzureTTSService and fixes
the race condition that causes scrambled TTS output across multiple
sentences.

## Features Added

- Change AzureTTSService to inherit from WordTTSService
- Subscribe to Azure SDK's synthesis_word_boundary event
- Emit word-level text with timing information via _words_queue
- Add synthesis lock for sequential sentence processing

## Race Condition Fix

Previously, each sentence's word boundary timestamps reset to 0,
causing downstream components to interleave words when reordering
frames by PTS. This resulted in scrambled output like:
  'Hello ! I What am questions AI have assistant...'

The fix adds cumulative audio offset tracking to ensure monotonically
increasing PTS across all sentences:
  Sentence 1: pts = 0.1s, 0.5s, 0.8s (cumulative at end: 0.8s)
  Sentence 2: pts = 0.9s, 1.2s, 1.5s (0.8s + relative offset)

## Key Changes

- _cumulative_audio_offset: tracks total audio duration
- _handle_word_boundary: adds cumulative offset to timestamps
- _handle_completed: accumulates audio duration for next sentence
- flush_audio: resets cumulative offset at end of LLM response
- _handle_interruption: resets state on user interruption
- run_tts: uses synthesis lock for sequential processing

Fixes #2918

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-31 18:49:48 +09:00
Aleix Conchillo Flaqué
cbdc2b7d2d Merge pull request #3330 from pipecat-ai/aleix/update-turn-start-strategies-deprecations 
update turn start strategies deprecations
 2025-12-30 21:04:47 -08:00
Aleix Conchillo Flaqué
66a9dc70c7 LLMUserAggregator: fix turn strategies renaming 2025-12-30 20:59:48 -08:00
Aleix Conchillo Flaqué
846ca500d3 turns: update old turn_start_strategies deprecations 2025-12-30 19:50:10 -08:00
Mark Backman
2e700c8576 Merge pull request #3324 from pipecat-ai/mb/bump-small-webrtc-prebuilt-version 
Bump small-webrtc-prebuilt verison to 2.0.4, update uv.lock
 2025-12-30 20:10:11 -05:00
Aleix Conchillo Flaqué
fd2efb3b3a Merge pull request #3325 from pipecat-ai/aleix/rename-bot-turn-start-to-user-turn-stop 
turns: rename bot turn start to user turn stop strategies
 2025-12-30 14:36:02 -08:00
Aleix Conchillo Flaqué
eb5a797b12 turns: rename bot turn start to user turn stop strategies 2025-12-30 14:33:58 -08:00
Mark Backman
f4626a4fc4 Bump small-webrtc-prebuilt verison to 2.0.4, update uv.lock 2025-12-30 14:19:20 -05:00
Aleix Conchillo Flaqué
fb9a772e33 Merge pull request #3319 from pipecat-ai/aleix/openaillmcontext-backwards-compatibility 
BaseInputTransport: fix OpenAILLMContext backwards compatibility
 2025-12-30 09:35:43 -08:00
Aleix Conchillo Flaqué
4630e76942 ExternalUserTurnStartStrategy: disable interruptions 2025-12-30 09:32:31 -08:00
Aleix Conchillo Flaqué
4dba9ea329 BaseInputTransport: fix OpenAILLMContext backwards compatibility 2025-12-30 09:32:31 -08:00
Mark Backman
233bc23bf9 Merge pull request #3320 from joshwhiton/fix-changelog-numba-pin 
Fix numba pin wording in changelog
 2025-12-30 08:50:06 -05:00
Mark Backman
9c6d0f1be1 Merge pull request #3322 from gui217/fix/rnnoise_filter_handle_empty_audio 
Fix/rnnoise filter handle empty audio
 2025-12-30 08:39:42 -05:00
gui217
32c3298eff Apply ruff formatting to test file 2025-12-30 13:39:36 +02:00
gui217
ec5fb392c4 Clean up test comments after rnnoise filter fix 2025-12-30 13:35:13 +02:00
gui217
bad8f8aa51 Fix rnnoise filter to handle empty audio 2025-12-30 13:32:36 +02:00
joshwhiton
6a7b6bcded Fix numba pin wording in changelog 2025-12-30 11:26:22 +07:00
Aleix Conchillo Flaqué
00548769cb Merge pull request #3318 from pipecat-ai/aleix/llm-user-aggregator-allow-interruptions 
LLMUserAggregator: also read deprecated allow_interruptions
 2025-12-29 18:11:57 -08:00
Aleix Conchillo Flaqué
0a0ab51cc7 LLMUserAggregator: also read deprecated allow_interruptions 2025-12-29 17:57:18 -08:00
Mark Backman
8339c2c2c7 Merge pull request #3317 from pipecat-ai/mb/add-changelog-other 
Add 'other' changelog category
 2025-12-29 20:46:18 -05:00
Aleix Conchillo Flaqué
ad4c22cf44 Merge pull request #3316 from pipecat-ai/aleix/llm-user-aggreagtor-enable-interruptions 
turns(user): add support for enabling/disabling interruptions
 2025-12-29 17:45:56 -08:00
Mark Backman
8ac6421988 Add 'other' changelog category 2025-12-29 20:43:24 -05:00
Aleix Conchillo Flaqué
9fe99ed880 add and update changelog entries 2025-12-29 17:35:10 -08:00
Aleix Conchillo Flaqué
97ab0d4f53 examples: added 52-live-translation without interruptions 2025-12-29 17:30:06 -08:00
Aleix Conchillo Flaqué
ffbbb1b3f5 turns(user): add support for enabling/disabling interruptions 2025-12-29 17:00:03 -08:00
Aleix Conchillo Flaqué
e22a6c9e4d Merge pull request #3305 from omChauhanDev/fix_unregister_function 
fix: missing key access by adding existence check
 2025-12-29 14:52:41 -08:00
Aleix Conchillo Flaqué
09e79149ea Merge pull request #3310 from omChauhanDev/fix-task-manager 
fix: preserve asyncio.Task return value in create_task
 2025-12-29 14:43:24 -08:00
Aleix Conchillo Flaqué
c799d63f8c Merge pull request #3308 from pipecat-ai/aleix/external-turn-start-strategies 
turns: add external user and bot turn start strategies
 2025-12-29 14:42:38 -08:00
Aleix Conchillo Flaqué
bd9a316d7a transports: don't use interruptions_allowed to avoid deprecation warning 2025-12-29 14:40:00 -08:00
Aleix Conchillo Flaqué
c8f47b4b22 turns: add UserTurnStartedParams and BotTurnStartedParams 2025-12-29 14:32:08 -08:00
Aleix Conchillo Flaqué
cf46431d92 update changelog file 2025-12-29 10:30:41 -08:00
Mark Backman
c28ed2206c DeepgramSTTService pushes user started/stopped speaking and interruption frames 2025-12-29 10:17:35 -08:00
Mark Backman
30e6a33930 Update VoicemailDetector to use ExternalTurnStartStrategies 2025-12-29 10:17:35 -08:00
Aleix Conchillo Flaqué
46db8e58d6 LLMUserAggregator: fix backwards compatibility with ExternalTurnStartStrategies 2025-12-29 10:17:35 -08:00
Aleix Conchillo Flaqué
e757b4bf6f tests: added external user and bot turn start strategies 2025-12-29 10:17:35 -08:00
Aleix Conchillo Flaqué
c821e9f8fd turns: add external user and bot turn start strategies 
External strategies are strategies where the logic for user turn start and turn
end come from a different processors (e.g. an STT).
 2025-12-29 10:17:35 -08:00
Mark Backman
01ce06c756 Merge pull request #3288 from pipecat-ai/mb/inworld-cleanup 
Inworld TTS service clean up
 2025-12-29 13:07:20 -05:00
Mark Backman
4bc490c843 Merge pull request #3289 from pipecat-ai/mb/audio-context-tts-service-base 
Add AudioContextTTSService base class, update AudioContextWordTTSServ…
 2025-12-29 13:05:06 -05:00
Mark Backman
345885fe7d Merge pull request #3271 from pipecat-ai/mb/changelog-3268 
Update fragment name for 3268
 2025-12-29 13:04:03 -05:00
Mark Backman
6475077fc8 Merge pull request #3313 from pipecat-ai/mb/ultravox-s2s-readme 
Update Ultravox README link
 2025-12-29 13:03:39 -05:00
Mark Backman
d646ca594b Update Ultravox README link 2025-12-29 11:43:28 -05:00
Mark Backman
7c0d897aa3 Merge pull request #3300 from omChauhanDev/nvidia-expose-use_ssl-param 
exposed use_ssl param in nvidia services
 2025-12-29 09:18:26 -05:00
Aleix Conchillo Flaqué
0e8e3afc85 Merge pull request #3307 from pipecat-ai/aleix/simplify-turns-package-imports 
turns: simplify imports and don't require full strategy module path
 2025-12-28 18:51:23 -08:00
Aleix Conchillo Flaqué
db85043841 Merge pull request #3297 from pipecat-ai/aleix/deprecate-allow-interruptions 
deprecate allow interruptions
 2025-12-28 18:50:15 -08:00
Om Chauhan
a181e01310 fixed: create_task to return coroutine result 2025-12-29 07:46:15 +05:30
Aleix Conchillo Flaqué
5496aa722f turns: simplify imports and don't require full strategy module path 2025-12-28 16:20:15 -08:00
Aleix Conchillo Flaqué
053f59ed6e FrameProcessor: deprecated interruptions_allowed 2025-12-28 08:27:02 -08:00
Aleix Conchillo Flaqué
5b93fb9609 PipelineTask: deprecate allow_interruptions parameter 2025-12-28 08:27:02 -08:00