Add documentation for AI 视频助手 including configuration, features, integrations, and quickstart guide
- Create a new JSON schema file for documentation settings. - Add core features documentation detailing assistant configuration and model integration. - Include an index page introducing the AI 视频助手 and its platform components. - Document integration methods for model services and external agents. - Provide a quickstart guide for setting up the local development environment using Docker. - Add light and dark logo SVG files for branding purposes.
This commit is contained in:
Xin Wang
57
docs/docs.json
Normal file
57
docs/docs.json
Normal file
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"$schema": "https://mintlify.com/docs.json",
|
||||
"name": "AI 视频助手",
|
||||
"theme": "mint",
|
||||
"colors": {
|
||||
"primary": "#1b2741",
|
||||
"light": "#3a4a6b",
|
||||
"dark": "#0c1426"
|
||||
},
|
||||
"logo": {
|
||||
"light": "/logo-light.svg",
|
||||
"dark": "/logo-dark.svg",
|
||||
"href": "/"
|
||||
},
|
||||
"appearance": {
|
||||
"default": "dark"
|
||||
},
|
||||
"fonts": {
|
||||
"family": "Inter",
|
||||
"weight": 400,
|
||||
"heading": {
|
||||
"family": "Cormorant Garamond",
|
||||
"weight": 300
|
||||
}
|
||||
},
|
||||
"icons": {
|
||||
"library": "lucide"
|
||||
},
|
||||
"background": {
|
||||
"decoration": "gradient",
|
||||
"color": {
|
||||
"light": "#f3f5fb",
|
||||
"dark": "#070b16"
|
||||
}
|
||||
},
|
||||
"styling": {
|
||||
"eyebrows": "breadcrumbs",
|
||||
"codeblocks": {
|
||||
"theme": {
|
||||
"light": "github-light",
|
||||
"dark": "github-dark"
|
||||
}
|
||||
}
|
||||
},
|
||||
"navigation": {
|
||||
"groups": [
|
||||
{
|
||||
"group": "开始使用",
|
||||
"pages": ["index", "quickstart"]
|
||||
},
|
||||
{
|
||||
"group": "产品与对接",
|
||||
"pages": ["features", "integrations"]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
31
docs/features.mdx
Normal file
31
docs/features.mdx
Normal file
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: 核心功能
|
||||
description: 项目的主要能力与配置关系。
|
||||
icon: bot
|
||||
---
|
||||
|
||||
# 核心功能
|
||||
|
||||
## 助手配置
|
||||
|
||||
助手是运行时入口，支持提示词、工作流和外部托管三类大脑：
|
||||
|
||||
| 类型 | 用途 |
|
||||
| --- | --- |
|
||||
| `prompt` | 用提示词驱动的基础对话，可绑定工具。 |
|
||||
| `workflow` | 用节点图编排对话流程。 |
|
||||
| `dify` / `fastgpt` / `opencode` | 将对话处理交给外部 Agent 服务。 |
|
||||
|
||||
其中 `prompt` 和 `workflow` 支持 `pipeline` 与 `realtime`；外部托管 Agent 仅支持 `pipeline`。
|
||||
|
||||
## 模型与知识库
|
||||
|
||||
模型资源按具体协议注册，而不是按厂商名称猜测。当前支持 OpenAI 兼容的 LLM、ASR、TTS、Embedding、Realtime，也支持讯飞 ASR/TTS/SuperTTS、StepFun Realtime，以及 Dify、FastGPT、OpenCode Agent。
|
||||
|
||||
知识库可绑定一个 Embedding 模型资源。助手引用知识库后，需先解绑才能删除该知识库。
|
||||
|
||||
## 工具与历史
|
||||
|
||||
- 可复用工具支持结束通话和 HTTP 请求两种类型。
|
||||
- HTTP 工具的 URL、请求头、参数和密钥独立管理。
|
||||
- 历史记录保留会话、通道、运行模式及消息序列，便于排查对话。
|
||||
26
docs/index.mdx
Normal file
26
docs/index.mdx
Normal file
@@ -0,0 +1,26 @@
|
||||
---
|
||||
title: AI 视频助手
|
||||
description: 面向实时语音交互的助手管理与集成平台。
|
||||
icon: video
|
||||
---
|
||||
|
||||
# AI 视频助手
|
||||
|
||||
在一个界面中配置模型、知识库、工具和助手，并通过 WebRTC 或 WebSocket 接入实时语音对话。
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="快速开始" icon="rocket" color="#3a4a6b" href="/quickstart">
|
||||
本地启动服务并完成首次助手配置。
|
||||
</Card>
|
||||
<Card title="对接说明" icon="plug" color="#3a4a6b" href="/integrations">
|
||||
了解模型、外部 Agent 与语音通道的接入方式。
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## 平台组成
|
||||
|
||||
- **助手**：定义提示词、运行模式、模型绑定及可选工作流。
|
||||
- **组件库**：统一维护模型资源、知识库和可复用工具。
|
||||
- **语音引擎**：Pipecat 管线支持浏览器 WebRTC 与裸 WebSocket 音频流。
|
||||
|
||||
> 凭证只保存在模型资源或助手的服务端配置中；接口返回会自动打码。
|
||||
48
docs/integrations.mdx
Normal file
48
docs/integrations.mdx
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
title: 对接说明
|
||||
description: 模型服务、外部 Agent 与实时语音通道的接入方式。
|
||||
icon: plug
|
||||
---
|
||||
|
||||
# 对接说明
|
||||
|
||||
## 模型服务
|
||||
|
||||
优先通过「组件库 / 模型资源」接入。接口定义会动态给出表单字段，凭证写入 `secrets`，普通参数写入 `values`。
|
||||
|
||||
| 场景 | 推荐接口类型 |
|
||||
| --- | --- |
|
||||
| 通用大模型、转写、合成、向量 | `openai-llm`、`openai-asr`、`openai-tts`、`openai-embedding` |
|
||||
| 端到端实时语音 | `stepfun-realtime` 或 `openai-realtime` |
|
||||
| 讯飞语音 | `xfyun-asr`、`xfyun-tts`、`xfyun-super-tts` |
|
||||
|
||||
讯飞接入需要将 `appId`、`apiKey`、`apiSecret` 写入 `secrets`，WebSocket 地址及音色等参数写入 `values`。
|
||||
|
||||
## 外部 Agent
|
||||
|
||||
创建助手时选择 Dify、FastGPT 或 OpenCode：
|
||||
|
||||
- Dify：填写服务地址和 API Key。
|
||||
- FastGPT：额外填写应用 ID。
|
||||
- OpenCode：填写服务地址、API Key 和可选提示词。
|
||||
|
||||
外部 Agent 的上下文、知识库和工具由对方服务处理。
|
||||
|
||||
## 实时语音通道
|
||||
|
||||
| 通道 | 地址 | 适用场景 |
|
||||
| --- | --- | --- |
|
||||
| WebRTC 信令 | `ws(s)://<host>/ws/voice` | 浏览器低延迟语音与视频输入。 |
|
||||
| 裸音频 WebSocket | `ws(s)://<host>/ws/stream` | 服务端、话务网关或自定义客户端。 |
|
||||
|
||||
两个通道均使用管理员登录 Cookie 鉴权。推荐仅传 `assistant_id`，让服务端解析模型密钥；`inline_config` 仅用于调试。
|
||||
|
||||
### WebSocket 音频流启动
|
||||
|
||||
连接 `/ws/stream` 后，先发送一条文本 JSON，再发送二进制音频帧：
|
||||
|
||||
```json
|
||||
{"assistant_id":"asst_xxx"}
|
||||
```
|
||||
|
||||
WebRTC 在公网环境通常需要 TURN。设置 `TURN_URLS` 和 `TURN_SECRET` 后，可通过 `GET /api/webrtc/ice-servers` 获取浏览器 ICE 配置。
|
||||
18
docs/logo-dark.svg
Normal file
18
docs/logo-dark.svg
Normal file
@@ -0,0 +1,18 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 196 48" fill="none">
|
||||
<defs>
|
||||
<radialGradient id="sky" cx="0" cy="0" r="1" gradientTransform="matrix(31 0 0 31 14 10)" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#5F86B8" stop-opacity=".75"/>
|
||||
<stop offset="1" stop-color="#5F86B8" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
<radialGradient id="lavender" cx="0" cy="0" r="1" gradientTransform="matrix(28 0 0 28 38 39)" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#8A78AD" stop-opacity=".68"/>
|
||||
<stop offset="1" stop-color="#8A78AD" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
</defs>
|
||||
<circle cx="24" cy="24" r="24" fill="#E8EDF9"/>
|
||||
<circle cx="24" cy="24" r="24" fill="url(#sky)"/>
|
||||
<circle cx="24" cy="24" r="24" fill="url(#lavender)"/>
|
||||
<rect x="14" y="17.5" width="14" height="13" rx="2.25" stroke="#0C1426" stroke-width="2.5"/>
|
||||
<path d="m28 21.25 7-4v13.5l-7-4z" stroke="#0C1426" stroke-width="2.5" stroke-linejoin="round"/>
|
||||
<text x="61" y="31" fill="#E8EDF9" font-family="Cormorant Garamond, serif" font-size="24" font-weight="400">AI视频助手</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.1 KiB |
18
docs/logo-light.svg
Normal file
18
docs/logo-light.svg
Normal file
@@ -0,0 +1,18 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 196 48" fill="none">
|
||||
<defs>
|
||||
<radialGradient id="sky" cx="0" cy="0" r="1" gradientTransform="matrix(31 0 0 31 14 10)" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#A8C8E8" stop-opacity=".8"/>
|
||||
<stop offset="1" stop-color="#A8C8E8" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
<radialGradient id="lavender" cx="0" cy="0" r="1" gradientTransform="matrix(28 0 0 28 38 39)" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#C8B8E0" stop-opacity=".72"/>
|
||||
<stop offset="1" stop-color="#C8B8E0" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
</defs>
|
||||
<circle cx="24" cy="24" r="24" fill="#E8EDF9"/>
|
||||
<circle cx="24" cy="24" r="24" fill="url(#sky)"/>
|
||||
<circle cx="24" cy="24" r="24" fill="url(#lavender)"/>
|
||||
<rect x="14" y="17.5" width="14" height="13" rx="2.25" stroke="#0C1426" stroke-width="2.5"/>
|
||||
<path d="m28 21.25 7-4v13.5l-7-4z" stroke="#0C1426" stroke-width="2.5" stroke-linejoin="round"/>
|
||||
<text x="61" y="31" fill="#0F1B33" font-family="Cormorant Garamond, serif" font-size="24" font-weight="400">AI视频助手</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.1 KiB |
45
docs/quickstart.mdx
Normal file
45
docs/quickstart.mdx
Normal file
@@ -0,0 +1,45 @@
|
||||
---
|
||||
title: 快速开始
|
||||
description: 用 Docker 启动本地开发环境，并创建第一个可语音预览的助手。
|
||||
icon: rocket
|
||||
---
|
||||
|
||||
# 快速开始
|
||||
|
||||
## 1. 启动服务
|
||||
|
||||
在项目根目录执行：
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
make db-seed
|
||||
```
|
||||
|
||||
首次启动会构建后端镜像。启动后访问：
|
||||
|
||||
- 前端：`http://localhost:3030`
|
||||
- 后端健康检查：`http://localhost:8000/health`
|
||||
- 后端接口文档：`http://localhost:8000/docs`
|
||||
|
||||
## 2. 配置模型资源
|
||||
|
||||
进入「组件库 / 模型资源」，选择协议类型并填写连接信息。OpenAI 兼容服务通常需要：
|
||||
|
||||
- `values.modelId`：模型标识
|
||||
- `values.apiUrl`：服务地址
|
||||
- `secrets.apiKey`：访问密钥
|
||||
|
||||
保存前可使用“测试连接”确认可用性。
|
||||
|
||||
## 3. 创建助手
|
||||
|
||||
进入「创建助手」，绑定需要的模型资源：
|
||||
|
||||
- **pipeline**：绑定 LLM、ASR、TTS，适合常规语音对话。
|
||||
- **realtime**：绑定 Realtime 模型，适合端到端语音交互。
|
||||
|
||||
保存后，从助手详情页打开语音预览即可测试。
|
||||
|
||||
<Note>
|
||||
浏览器麦克风仅在 localhost 或 HTTPS 下可用。局域网、远程环境请按仓库中的 `deploy/README.md` 配置 HTTPS 与 TURN。
|
||||
</Note>
|
||||
Reference in New Issue
Block a user