4c05131536
- Revised mkdocs.yml to reflect the new site name and description, enhancing clarity for users. - Added a changelog.md to document important changes and updates for the project. - Introduced a roadmap.md to outline development plans and progress for future releases. - Expanded index.md with a comprehensive overview of the platform, including core features and installation instructions. - Enhanced concepts documentation with detailed explanations of assistants, engines, and their configurations. - Updated configuration documentation to provide clear guidance on environment setup and service configurations. - Added extra JavaScript for improved user experience in the documentation site.
223 lines
4.7 KiB
Markdown
223 lines
4.7 KiB
Markdown
<h1 align="center">Realtime Agent Studio</h1>
|
||
|
||
<p align="center">
|
||
<strong>构建实时交互音视频智能体的开源工作平台</strong>
|
||
</p>
|
||
|
||
<p align="center">
|
||
<img src="https://img.shields.io/badge/version-0.1.0-blue" alt="Version">
|
||
<img src="https://img.shields.io/badge/license-MIT-green" alt="License">
|
||
<img src="https://img.shields.io/badge/python-3.10+-blue" alt="Python">
|
||
<img src="https://img.shields.io/badge/node-18+-green" alt="Node">
|
||
</p>
|
||
|
||
<p align="center">
|
||
<a href="quickstart/index.md">快速开始</a> ·
|
||
<a href="api-reference/index.md">API 文档</a> ·
|
||
<a href="deployment/index.md">部署指南</a> ·
|
||
<a href="roadmap.md">路线图</a>
|
||
</p>
|
||
|
||
---
|
||
|
||
## 什么是 Realtime Agent Studio？
|
||
|
||
Realtime Agent Studio (RAS) 是一款以大语言模型为核心，构建实时交互音视频智能体的工作平台。支持管线式的全双工交互引擎和原生多模态模型两种架构，覆盖实时交互智能体的配置、测试、发布、监控全流程。
|
||
|
||
可以将 RAS 看作 [Vapi](https://vapi.ai)、[Retell](https://retellai.com)、[ElevenLabs Agents](https://elevenlabs.io) 的**开源替代方案**。
|
||
|
||
|
||
|
||
---
|
||
|
||
## 核心特性
|
||
|
||
<div class="grid cards" markdown>
|
||
|
||
- :zap: **低延迟实时引擎**
|
||
|
||
---
|
||
|
||
管线式全双工架构，ASR/LLM/TTS 流水线处理，支持智能打断，端到端延迟 < 500ms
|
||
|
||
- :brain: **多模态模型支持**
|
||
|
||
---
|
||
|
||
支持 GPT-4o Realtime、Gemini Live、Step Audio 等原生多模态模型直连
|
||
|
||
- :wrench: **可视化配置**
|
||
|
||
---
|
||
|
||
无代码配置助手、提示词、工具调用、知识库关联，所见即所得
|
||
|
||
- :electric_plug: **开放 API**
|
||
|
||
---
|
||
|
||
标准 WebSocket 协议，RESTful 管理接口，支持 Webhook 回调
|
||
|
||
- :shield: **私有化部署**
|
||
|
||
---
|
||
|
||
Docker 一键部署，数据完全自主可控，支持本地模型
|
||
|
||
- :chart_with_upwards_trend: **全链路监控**
|
||
|
||
---
|
||
|
||
完整会话回放，实时仪表盘，自动化测试与效果评估
|
||
|
||
</div>
|
||
|
||
---
|
||
|
||
## 系统架构
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph Client["客户端"]
|
||
Web[Web 浏览器]
|
||
App[移动应用]
|
||
SDK[SDK]
|
||
end
|
||
|
||
subgraph RAS["Realtime Agent Studio"]
|
||
Engine[实时交互引擎]
|
||
API[API 服务]
|
||
DB[(数据库)]
|
||
end
|
||
|
||
subgraph Pipeline["管线式引擎"]
|
||
ASR[语音识别]
|
||
LLM[大语言模型]
|
||
TTS[语音合成]
|
||
end
|
||
|
||
subgraph External["外部服务"]
|
||
OpenAI[OpenAI]
|
||
Azure[Azure]
|
||
Local[本地模型]
|
||
end
|
||
|
||
Client -->|WebSocket| Engine
|
||
Client -->|REST| API
|
||
Engine --> Pipeline
|
||
Engine <--> API
|
||
API <--> DB
|
||
Pipeline --> External
|
||
```
|
||
|
||
---
|
||
|
||
## 技术栈
|
||
|
||
| 层级 | 技术 |
|
||
|------|------|
|
||
| **前端** | React 18, TypeScript, Tailwind CSS, Zustand |
|
||
| **后端** | FastAPI (Python 3.10+) |
|
||
| **引擎** | Python, WebSocket, asyncio |
|
||
| **数据库** | SQLite / PostgreSQL |
|
||
| **部署** | Docker, Nginx |
|
||
|
||
---
|
||
|
||
## 快速导航
|
||
|
||
<div class="grid cards" markdown>
|
||
|
||
- :rocket: **[快速开始](quickstart/index.md)**
|
||
|
||
---
|
||
|
||
5 分钟创建你的第一个 AI 助手
|
||
|
||
- :book: **[核心概念](concepts/index.md)**
|
||
|
||
---
|
||
|
||
了解助手、管线、多模态等核心概念
|
||
|
||
- :wrench: **[安装部署](getting-started/index.md)**
|
||
|
||
---
|
||
|
||
环境准备与本地开发配置
|
||
|
||
- :robot: **[助手管理](assistants/index.md)**
|
||
|
||
---
|
||
|
||
创建和配置智能对话助手
|
||
|
||
- :gear: **[功能定制](customization/knowledge-base.md)**
|
||
|
||
---
|
||
|
||
知识库、工具、语音、工作流
|
||
|
||
- :bar_chart: **[数据分析](analysis/dashboard.md)**
|
||
|
||
---
|
||
|
||
仪表盘、历史记录、测试评估
|
||
|
||
- :electric_plug: **[API 参考](api-reference/index.md)**
|
||
|
||
---
|
||
|
||
WebSocket 协议与 REST 接口文档
|
||
|
||
- :cloud: **[部署指南](deployment/index.md)**
|
||
|
||
---
|
||
|
||
Docker 与生产环境部署
|
||
|
||
</div>
|
||
|
||
---
|
||
|
||
## 快速体验
|
||
|
||
### 使用 Docker 启动
|
||
|
||
```bash
|
||
git clone https://github.com/your-org/AI-VideoAssistant.git
|
||
cd AI-VideoAssistant
|
||
docker-compose up -d
|
||
```
|
||
|
||
访问 `http://localhost:3000` 即可使用控制台。
|
||
|
||
### WebSocket 连接示例
|
||
|
||
```javascript
|
||
const ws = new WebSocket('ws://localhost:8000/ws?assistant_id=YOUR_ID');
|
||
|
||
ws.onopen = () => {
|
||
ws.send(JSON.stringify({
|
||
type: 'session.start',
|
||
audio: { encoding: 'pcm_s16le', sample_rate_hz: 16000, channels: 1 }
|
||
}));
|
||
};
|
||
```
|
||
|
||
---
|
||
|
||
## 参与贡献
|
||
|
||
我们欢迎社区贡献！查看 [贡献指南](https://github.com/your-org/AI-VideoAssistant/blob/main/CONTRIBUTING.md) 了解如何参与。
|
||
|
||
- :star: Star 项目支持我们
|
||
- :bug: 提交 Issue 报告问题
|
||
- :hammer: 提交 PR 贡献代码
|
||
|
||
---
|
||
|
||
## 许可证
|
||
|
||
本项目基于 [MIT 许可证](https://github.com/your-org/AI-VideoAssistant/blob/main/LICENSE) 开源。
|