Add support for Xfyun ASR and TTS services in the backend

- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities.
- Update AssistantConfig model to include interface types for STT and TTS.
- Enhance credential testing to validate Xfyun credentials.
- Modify service factory to create Xfyun services based on configuration.
- Update README with new configuration details for Xfyun integration.
- Add new frontend components for visualizing audio streams and managing user interactions.
This commit is contained in:
Xin Wang
2026-06-11 10:51:08 +08:00
parent c69dec04e0
commit e25dfd4003
19 changed files with 1595 additions and 71 deletions

View File

@@ -55,6 +55,18 @@ ai-video-backend/
| STT | SenseVoice / FunASR | 本地 OpenAI 兼容转写服务 |
| TTS | CosyVoice | 本地 OpenAI 兼容 TTS 服务 |
### 讯飞 ASR / TTS / SuperTTS
讯飞继续复用 `ProviderCredential` 的现有字段,不增加专属列:
- `interface_type`: `xfyun`
- `api_url`: 讯飞 WebSocket URL`https://` 会自动转为 `wss://`
- `api_key`: `{"appId":"...","apiKey":"...","apiSecret":"..."}`
- ASR `model_id`: `iat`
- 普通 TTS `model_id`: `tts`
- 超拟人 TTS `model_id`: `supertts`(包含 `/private/` 的 URL 也会自动识别)
- TTS `voice`: 讯飞音色 ID`speed=1.0` 对应讯飞正常语速 `50`
## 本地运行(用 uv,Python 3.12)
```bash
@@ -97,7 +109,7 @@ docker compose --profile remote up -d
## 待联调 / TODO
- [ ] `pip install` 后跑通,核对 pipecat 版本的服务/transport 构造参数(代码内有注释)
- [ ] 联调 Pipecat 1.3.0 语音链路与各 OpenAI 兼容服务
- [ ] 起本地 SenseVoice / CosyVoice 的 OpenAI 兼容服务
- [ ] `realtime` 模式(目前只 `pipeline` 级联)
- [x] 前端 `DebugVoicePanel``/ws/voice`(参考 dograh `useWebSocketRTC.tsx`)