Refactor backend to support interface-definition driven model resources
- Introduce a new model structure for managing interface definitions and model resources, enhancing the backend's capability to handle various service integrations. - Update the Makefile to reflect changes in database seeding and resource management commands. - Remove the deprecated credentials management routes and replace them with a unified model registry API. - Modify existing routes and schemas to align with the new model structure, ensuring seamless integration with the frontend. - Enhance database seeding scripts to populate new model resources and their configurations. - Update README documentation to reflect the new architecture and usage instructions for model resources and interface definitions.
This commit is contained in:
@@ -25,7 +25,7 @@ pipecat 把"管线"和"输出方式"解耦:同一条 `STT→LLM→TTS` 管线可
|
||||
```
|
||||
ai-video-backend/
|
||||
├── app.py # FastAPI 入口,挂路由 + CORS
|
||||
├── config.py # 读 .env,所有 provider 接入点
|
||||
├── config.py # 读 .env,模型接口环境变量兜底
|
||||
├── models.py # AssistantConfig(对齐前端 AssistantForm)
|
||||
├── routes/ # 一个文件一组端点(对齐 dograh routes/)
|
||||
│ ├── health.py
|
||||
@@ -33,7 +33,7 @@ ai-video-backend/
|
||||
│ └── voice_ws.py # WS 裸音频流
|
||||
├── services/
|
||||
│ └── pipecat/ # 引擎(对齐 dograh services/pipecat/)
|
||||
│ ├── service_factory.py # 建 STT/LLM/TTS(加 provider 在此)
|
||||
│ ├── service_factory.py # 建 STT/LLM/TTS(按 interface_type 分发)
|
||||
│ ├── transports.py # transport 工厂(加输出方式在此)
|
||||
│ └── pipeline.py # 管线拼装与运行(transport 无关)
|
||||
├── Dockerfile
|
||||
@@ -57,15 +57,33 @@ ai-video-backend/
|
||||
|
||||
### 讯飞 ASR / TTS / SuperTTS
|
||||
|
||||
讯飞继续复用 `ProviderCredential` 的现有字段,不增加专属列:
|
||||
讯飞鉴权直接存入对应 `ModelResource.secrets`,接口参数存入 `ModelResource.values`:
|
||||
|
||||
- `interface_type`: `xfyun`
|
||||
- `api_url`: 讯飞 WebSocket URL(`https://` 会自动转为 `wss://`)
|
||||
- `api_key`: `{"appId":"...","apiKey":"...","apiSecret":"..."}`
|
||||
- ASR `model_id`: `iat`
|
||||
- 普通 TTS `model_id`: `tts`
|
||||
- 超拟人 TTS `model_id`: `supertts`(包含 `/private/` 的 URL 也会自动识别)
|
||||
- TTS `voice`: 讯飞音色 ID;`speed=1.0` 对应讯飞正常语速 `50`
|
||||
- 普通语音识别:`interface_type=xfyun-asr`
|
||||
- 普通语音合成:`interface_type=xfyun-tts`
|
||||
- 超拟人语音合成:`interface_type=xfyun-super-tts`
|
||||
- `values.apiUrl` 保存讯飞 WebSocket URL,音色、语速等可选参数也放在 `values`
|
||||
- `secrets` 分别保存 `appId`、`apiKey`、`apiSecret`
|
||||
|
||||
## 接口定义驱动的模型注册表
|
||||
|
||||
LLM、ASR、TTS、Embedding、Realtime 使用同一套两层结构:
|
||||
|
||||
```text
|
||||
assistant_model_bindings -> model_resources -> interface_definitions
|
||||
```
|
||||
|
||||
- `interface_definitions`: 定义具体接入协议、能力和动态表单字段。
|
||||
- `model_resources`: 每条资源自带 `values/secrets`,不复用供应商账号。
|
||||
- `assistant_model_bindings`: 助手按能力选择模型资源。
|
||||
|
||||
`interface_type` 是具体协议,例如 `xfyun-asr`、`xfyun-tts`、
|
||||
`xfyun-super-tts`,后端严格按它选择服务实现,不根据模型 ID 或 URL 猜测。
|
||||
|
||||
API:
|
||||
|
||||
- `/api/interface-definitions`: 前端读取字段定义并动态生成 Dialog。
|
||||
- `/api/model-resources`: 统一模型资源 CRUD,敏感字段逐项打码。
|
||||
|
||||
## 本地运行(用 uv,Python 3.12)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user