Implement StepFun Realtime service and enhance AssistantConfig

- Add new fields to AssistantConfig for realtime interface configuration, including types, values, and secrets.
- Introduce StepFunRealtimeService to handle speech-to-speech processing via WebSocket, integrating STT, LLM, and TTS functionalities.
- Refactor pipeline execution to support a new realtime mode, allowing direct text input processing and immediate responses.
- Update model resource testing to include validation for StepFun Realtime connections.
- Enhance service factory to create realtime services based on configuration settings.
- Modify README documentation to reflect new realtime capabilities and usage instructions.
This commit is contained in:
Xin Wang
2026-06-14 23:41:40 +08:00
parent d55b87cfbf
commit 0309c154b5
11 changed files with 612 additions and 19 deletions

View File

@@ -78,6 +78,25 @@ INTERFACE_DEFINITIONS: list[dict] = [
"capability": "Realtime",
"fields": OPENAI_COMMON + [field("voice", "Voice")],
},
{
"interface_type": "stepfun-realtime",
"name": "StepFun StepAudio Realtime",
"capability": "Realtime",
"fields": OPENAI_COMMON
+ [
field("voice", "Voice", required=True, default="linjiajiejie"),
field("inputSampleRate", "Input Sample Rate", type_="number", default=24000),
field("outputSampleRate", "Output Sample Rate", type_="number", default=24000),
field("prefixPaddingMs", "VAD Prefix Padding (ms)", type_="number", default=500),
field("silenceDurationMs", "VAD Silence Duration (ms)", type_="number", default=300),
field(
"energyAwakenessThreshold",
"VAD Energy Threshold",
type_="number",
default=2500,
),
],
},
{
"interface_type": "xfyun-asr",
"name": "Xfyun Streaming ASR",