Add support for Xfyun ASR and TTS services in the backend
- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities. - Update AssistantConfig model to include interface types for STT and TTS. - Enhance credential testing to validate Xfyun credentials. - Modify service factory to create Xfyun services based on configuration. - Update README with new configuration details for Xfyun integration. - Add new frontend components for visualizing audio streams and managing user interactions.
This commit is contained in:
@@ -55,6 +55,18 @@ ai-video-backend/
|
||||
| STT | SenseVoice / FunASR | 本地 OpenAI 兼容转写服务 |
|
||||
| TTS | CosyVoice | 本地 OpenAI 兼容 TTS 服务 |
|
||||
|
||||
### 讯飞 ASR / TTS / SuperTTS
|
||||
|
||||
讯飞继续复用 `ProviderCredential` 的现有字段,不增加专属列:
|
||||
|
||||
- `interface_type`: `xfyun`
|
||||
- `api_url`: 讯飞 WebSocket URL(`https://` 会自动转为 `wss://`)
|
||||
- `api_key`: `{"appId":"...","apiKey":"...","apiSecret":"..."}`
|
||||
- ASR `model_id`: `iat`
|
||||
- 普通 TTS `model_id`: `tts`
|
||||
- 超拟人 TTS `model_id`: `supertts`(包含 `/private/` 的 URL 也会自动识别)
|
||||
- TTS `voice`: 讯飞音色 ID;`speed=1.0` 对应讯飞正常语速 `50`
|
||||
|
||||
## 本地运行(用 uv,Python 3.12)
|
||||
|
||||
```bash
|
||||
@@ -97,7 +109,7 @@ docker compose --profile remote up -d
|
||||
|
||||
## 待联调 / TODO
|
||||
|
||||
- [ ] `pip install` 后跑通,核对 pipecat 版本的服务/transport 构造参数(代码内有注释)
|
||||
- [ ] 联调 Pipecat 1.3.0 语音链路与各 OpenAI 兼容服务
|
||||
- [ ] 起本地 SenseVoice / CosyVoice 的 OpenAI 兼容服务
|
||||
- [ ] `realtime` 模式(目前只 `pipeline` 级联)
|
||||
- [x] 前端 `DebugVoicePanel` 接 `/ws/voice`(参考 dograh `useWebSocketRTC.tsx`)
|
||||
|
||||
Reference in New Issue
Block a user