feat(assemblyai): add vad_threshold parameter for U3 Pro
Add vad_threshold parameter to AssemblyAIConnectionParams to support voice activity detection threshold configuration for the u3-rt-pro model. This parameter allows users to align AssemblyAI's VAD threshold with their external VAD systems (e.g., Silero VAD) to avoid the "dead zone" where AssemblyAI transcribes speech that the external VAD hasn't detected yet, which can delay interruption handling. - Range: 0.0 to 1.0 (lower = more sensitive) - Default: 0.3 (API default when not sent) - Only applicable to u3-rt-pro model - Automatically included in WebSocket query parameters Recommended usage: Set vad_threshold to match your VAD's activation threshold (e.g., both at 0.3) for optimal performance.
This commit is contained in:
@@ -127,8 +127,6 @@ class AssemblyAIConnectionParams(BaseModel):
|
||||
Parameters:
|
||||
sample_rate: Audio sample rate in Hz. Defaults to 16000.
|
||||
encoding: Audio encoding format. Defaults to "pcm_s16le".
|
||||
formatted_finals: Whether to enable transcript formatting. Defaults to True.
|
||||
word_finalization_max_wait_time: Maximum time to wait for word finalization in milliseconds.
|
||||
end_of_turn_confidence_threshold: Confidence threshold for end-of-turn detection.
|
||||
min_turn_silence: Minimum silence duration when confident about end-of-turn.
|
||||
min_end_of_turn_silence_when_confident: DEPRECATED. Use min_turn_silence instead.
|
||||
@@ -139,16 +137,24 @@ class AssemblyAIConnectionParams(BaseModel):
|
||||
language_detection: Enable automatic language detection. Only applicable to
|
||||
universal-streaming-multilingual. When enabled, Turn messages include
|
||||
language_code and language_confidence fields. Defaults to None (not sent).
|
||||
format_turns: Whether to format transcript turns. Defaults to True.
|
||||
format_turns: Whether to format transcript turns. Only applicable to
|
||||
universal-streaming-english and universal-streaming-multilingual models.
|
||||
For u3-rt-pro, formatting is automatic and built-in. Defaults to True.
|
||||
speaker_labels: Enable speaker diarization. When enabled, final transcripts
|
||||
(end_of_turn=True) include a speaker field identifying the speaker
|
||||
(e.g., "Speaker A", "Speaker B"). Defaults to None (not sent).
|
||||
vad_threshold: Voice activity detection confidence threshold. Only applicable to
|
||||
u3-rt-pro. The confidence threshold (0.0 to 1.0) for classifying audio frames
|
||||
as silence. Frames with VAD confidence below this value are considered silent.
|
||||
Increase for noisy environments to reduce false speech detection. Defaults to
|
||||
0.3 (API default). For best performance when using with external VAD (e.g., Silero),
|
||||
align this value with your VAD's activation threshold to avoid the "dead zone"
|
||||
where AssemblyAI transcribes speech that your VAD hasn't detected yet.
|
||||
Defaults to None (not sent).
|
||||
"""
|
||||
|
||||
sample_rate: int = 16000
|
||||
encoding: Literal["pcm_s16le", "pcm_mulaw"] = "pcm_s16le"
|
||||
formatted_finals: bool = True
|
||||
word_finalization_max_wait_time: Optional[int] = None
|
||||
end_of_turn_confidence_threshold: Optional[float] = None
|
||||
min_turn_silence: Optional[int] = None
|
||||
min_end_of_turn_silence_when_confident: Optional[int] = None # Deprecated
|
||||
@@ -161,6 +167,7 @@ class AssemblyAIConnectionParams(BaseModel):
|
||||
language_detection: Optional[bool] = None
|
||||
format_turns: bool = True
|
||||
speaker_labels: Optional[bool] = None
|
||||
vad_threshold: Optional[float] = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def handle_deprecated_param(self):
|
||||
|
||||
Reference in New Issue
Block a user