Add documentation for AI 视频助手 including configuration, features, integrations, and quickstart guide
- Create a new JSON schema file for documentation settings. - Add core features documentation detailing assistant configuration and model integration. - Include an index page introducing the AI 视频助手 and its platform components. - Document integration methods for model services and external agents. - Provide a quickstart guide for setting up the local development environment using Docker. - Add light and dark logo SVG files for branding purposes.
This commit is contained in:
48
docs/integrations.mdx
Normal file
48
docs/integrations.mdx
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
title: 对接说明
|
||||
description: 模型服务、外部 Agent 与实时语音通道的接入方式。
|
||||
icon: plug
|
||||
---
|
||||
|
||||
# 对接说明
|
||||
|
||||
## 模型服务
|
||||
|
||||
优先通过「组件库 / 模型资源」接入。接口定义会动态给出表单字段,凭证写入 `secrets`,普通参数写入 `values`。
|
||||
|
||||
| 场景 | 推荐接口类型 |
|
||||
| --- | --- |
|
||||
| 通用大模型、转写、合成、向量 | `openai-llm`、`openai-asr`、`openai-tts`、`openai-embedding` |
|
||||
| 端到端实时语音 | `stepfun-realtime` 或 `openai-realtime` |
|
||||
| 讯飞语音 | `xfyun-asr`、`xfyun-tts`、`xfyun-super-tts` |
|
||||
|
||||
讯飞接入需要将 `appId`、`apiKey`、`apiSecret` 写入 `secrets`,WebSocket 地址及音色等参数写入 `values`。
|
||||
|
||||
## 外部 Agent
|
||||
|
||||
创建助手时选择 Dify、FastGPT 或 OpenCode:
|
||||
|
||||
- Dify:填写服务地址和 API Key。
|
||||
- FastGPT:额外填写应用 ID。
|
||||
- OpenCode:填写服务地址、API Key 和可选提示词。
|
||||
|
||||
外部 Agent 的上下文、知识库和工具由对方服务处理。
|
||||
|
||||
## 实时语音通道
|
||||
|
||||
| 通道 | 地址 | 适用场景 |
|
||||
| --- | --- | --- |
|
||||
| WebRTC 信令 | `ws(s)://<host>/ws/voice` | 浏览器低延迟语音与视频输入。 |
|
||||
| 裸音频 WebSocket | `ws(s)://<host>/ws/stream` | 服务端、话务网关或自定义客户端。 |
|
||||
|
||||
两个通道均使用管理员登录 Cookie 鉴权。推荐仅传 `assistant_id`,让服务端解析模型密钥;`inline_config` 仅用于调试。
|
||||
|
||||
### WebSocket 音频流启动
|
||||
|
||||
连接 `/ws/stream` 后,先发送一条文本 JSON,再发送二进制音频帧:
|
||||
|
||||
```json
|
||||
{"assistant_id":"asst_xxx"}
|
||||
```
|
||||
|
||||
WebRTC 在公网环境通常需要 TURN。设置 `TURN_URLS` 和 `TURN_SECRET` 后,可通过 `GET /api/webrtc/ice-servers` 获取浏览器 ICE 配置。
|
||||
Reference in New Issue
Block a user