Add reusable tools and assistant bindings

- Introduce a new `Tool` model and `AssistantToolBinding` for managing reusable tools within the application.
- Implement CRUD operations for tools in the new `tools` route, allowing for the creation, retrieval, updating, and deletion of tools.
- Update the `Assistant` model to include a list of tool IDs, enabling assistants to utilize these tools.
- Enhance the backend routes to synchronize tool bindings with assistants, ensuring proper management of tool associations.
- Add frontend components for tool management, including a tool picker in the assistant configuration, improving user experience in tool selection.
- Create a mobile call page to facilitate video calls, integrating camera and microphone selection for enhanced communication capabilities.
- Update API definitions to include tool-related types and operations, ensuring consistency across the application.
- Add a migration script to create the necessary database tables for tools and bindings, supporting the new functionality.
This commit is contained in:
Xin Wang
2026-07-10 10:05:41 +08:00
parent 919325505a
commit 3ed9e1b388
14 changed files with 1815 additions and 22 deletions

View File

@@ -98,6 +98,19 @@ uv run --with-requirements requirements.txt \
**写浏览器地址栏里访问 nginx 的同源地址**。如果地址栏带端口,这里也必须带端口;
如果地址栏不带端口,这里也不带端口。不要写后端直连端口 `:8000`
Next dev 还会校验开发资源的来源。远程访问时,把浏览器地址栏里的 host 加到
`frontend/next.config.ts``allowedDevOrigins`,并重启前端 dev server:
```ts
// frontend/next.config.ts
const nextConfig = {
allowedDevOrigins: ["127.0.0.1", "<远程机器IP或域名>"],
};
```
如果漏掉这一步,页面 HTML 可能能打开,但客户端 JS 无法接管:按钮无响应,
列表一直停在 `正在加载`,同时 Next dev 内部资源可能返回 `403 Unauthorized`
```bash
cd frontend
npm install
@@ -152,27 +165,108 @@ docker run --rm --name ai-video-nginx \
https://<远程机器IP或域名>:18189
```
### 6. 远程语音 / WebRTC 注意事项
### 6. 部署 coturn(TURN 中继)
- 云安全组至少放行 `80/tcp``443/tcp`
- 如果 HTTPS 映射到 `18189`,云安全组要放行 `18189/tcp`
- 如果浏览器和后端不在同一内网,WebRTC 媒体链路通常需要 TURN。
- 启动 coturn:
如果浏览器和后端不在同一内网,WebRTC 媒体链路通常需要 TURN。项目里的
`docker-compose.yaml` 已经带了 coturn,用 `remote` profile 启动即可
```bash
# 项目根 .env:
# PUBLIC_IP=<远程机器公网IP>
# TURN_SECRET=<和 backend/.env 一致的密钥>
#### 6.1 配项目根目录 `.env`
docker compose --profile remote up -d coturn
项目根目录的 `.env` coturn 容器用:
```text
PUBLIC_IP=<远程机器公网IP>
TURN_SECRET=<一串足够长的随机密钥>
```
- 云安全组放行 `3478/tcp``3478/udp``49152-49200/udp`
- `backend/.env` 中配置:
示例:
```text
PUBLIC_IP=118.89.89.89
TURN_SECRET=change-me-to-a-long-random-string
```
#### 6.2 配后端 `backend/.env`
后端用同一个 `TURN_SECRET` 签发短时 TURN 凭证:
```text
TURN_URLS=turn:<远程机器公网IP>:3478?transport=udp,turn:<远程机器公网IP>:3478?transport=tcp
TURN_SECRET=<同上>
TURN_SECRET=<和项目根 .env 一致的密钥>
```
示例:
```text
TURN_URLS=turn:118.89.89.89:3478?transport=udp,turn:118.89.89.89:3478?transport=tcp
TURN_SECRET=change-me-to-a-long-random-string
```
改完 `backend/.env` 后,重启后端 `uvicorn`
#### 6.3 启动 coturn
```bash
docker compose --profile remote up -d coturn
```
查看状态和日志:
```bash
docker compose ps coturn
docker compose logs -f coturn
```
#### 6.4 放行端口
云安全组至少放行:
```text
80/tcp
443/tcp
3478/tcp
3478/udp
49152-49200/udp
```
如果 HTTPS 映射到 `18189`,还要放行:
```text
18189/tcp
```
如果远程机器启用了 `ufw`:
```bash
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 18189/tcp
sudo ufw allow 3478/tcp
sudo ufw allow 3478/udp
sudo ufw allow 49152:49200/udp
```
#### 6.5 验证后端已下发 TURN
后端和 nginx 都跑起来后:
```bash
curl -k https://<远程机器IP或域名>:18189/api/webrtc/ice-servers
```
应该能看到 `stun:``turn:`:
```json
{
"iceServers": [
{"urls": "stun:stun.l.google.com:19302"},
{
"urls": "turn:<远程机器公网IP>:3478?transport=udp",
"username": "...",
"credential": "..."
}
]
}
```
## 前端怎么连后端
@@ -217,6 +311,26 @@ Docker 容器里的 `127.0.0.1` 是容器自己,不是宿主机。远程开发
检查 nginx 配置里是否保留了 `Upgrade``Connection` 头,以及 `/ws/`
`proxy_read_timeout 3600s`。本仓库的两份开发 nginx 配置都已经包含。
### 页面一直正在加载,但 API 直接访问正常?
先确认前端 dev server 已用浏览器访问同源地址启动:
```bash
NEXT_PUBLIC_API_BASE_URL=https://<远程机器IP或域名>:18189 \
npm run dev -- --hostname 0.0.0.0
```
再确认 `frontend/next.config.ts``allowedDevOrigins` 包含同一个 host:
```ts
allowedDevOrigins: ["127.0.0.1", "<远程机器IP或域名>"],
```
修改 `NEXT_PUBLIC_API_BASE_URL``allowedDevOrigins` 后都必须重启
`npm run dev`。若 API 直接返回 `200`,但页面按钮无响应、弹窗打不开、列表
仍停在 `正在加载`,通常是 Next dev 的 `_next/*` 客户端资源被跨源保护拦截,
不是后端接口没有数据。
### API 请求跨域报错?
走 nginx 时前端和后端同源,通常不需要跨源。若你绕过 nginx 直接访问后端,