446 lines
8.6 KiB
Markdown
446 lines
8.6 KiB
Markdown
# 工具与自动测试 (Tools & Autotest) API
|
|
|
|
工具与自动测试 API 用于管理可用工具列表和自动测试功能。
|
|
|
|
## 基础信息
|
|
|
|
| 项目 | 值 |
|
|
|------|-----|
|
|
| Base URL | `/api/v1/tools` |
|
|
| 认证方式 | Bearer Token (预留) |
|
|
|
|
---
|
|
|
|
## 可用工具 (Tool Registry)
|
|
|
|
系统内置以下工具：
|
|
|
|
| 工具ID | 名称 | 说明 |
|
|
|--------|------|------|
|
|
| search | 网络搜索 | 搜索互联网获取最新信息 |
|
|
| calculator | 计算器 | 执行数学计算 |
|
|
| weather | 天气查询 | 查询指定城市的天气 |
|
|
| translate | 翻译 | 翻译文本到指定语言 |
|
|
| knowledge | 知识库查询 | 从知识库中检索相关信息 |
|
|
| code_interpreter | 代码执行 | 安全地执行Python代码 |
|
|
|
|
---
|
|
|
|
## API 端点
|
|
|
|
### 1. 获取可用工具列表
|
|
|
|
```http
|
|
GET /api/v1/tools/list
|
|
```
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"tools": {
|
|
"search": {
|
|
"name": "网络搜索",
|
|
"description": "搜索互联网获取最新信息",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"query": {"type": "string", "description": "搜索关键词"}
|
|
},
|
|
"required": ["query"]
|
|
}
|
|
},
|
|
"calculator": {
|
|
"name": "计算器",
|
|
"description": "执行数学计算",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"expression": {"type": "string", "description": "数学表达式，如: 2 + 3 * 4"}
|
|
},
|
|
"required": ["expression"]
|
|
}
|
|
},
|
|
"weather": {
|
|
"name": "天气查询",
|
|
"description": "查询指定城市的天气",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"city": {"type": "string", "description": "城市名称"}
|
|
},
|
|
"required": ["city"]
|
|
}
|
|
},
|
|
"translate": {
|
|
"name": "翻译",
|
|
"description": "翻译文本到指定语言",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"text": {"type": "string", "description": "要翻译的文本"},
|
|
"target_lang": {"type": "string", "description": "目标语言，如: en, ja, ko"}
|
|
},
|
|
"required": ["text", "target_lang"]
|
|
}
|
|
},
|
|
"knowledge": {
|
|
"name": "知识库查询",
|
|
"description": "从知识库中检索相关信息",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"query": {"type": "string", "description": "查询内容"},
|
|
"kb_id": {"type": "string", "description": "知识库ID"}
|
|
},
|
|
"required": ["query"]
|
|
}
|
|
},
|
|
"code_interpreter": {
|
|
"name": "代码执行",
|
|
"description": "安全地执行Python代码",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"code": {"type": "string", "description": "要执行的Python代码"}
|
|
},
|
|
"required": ["code"]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 2. 获取工具详情
|
|
|
|
```http
|
|
GET /api/v1/tools/list/{tool_id}
|
|
```
|
|
|
|
**Path Parameters:**
|
|
|
|
| 参数 | 类型 | 说明 |
|
|
|------|------|------|
|
|
| tool_id | string | 工具ID |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"name": "计算器",
|
|
"description": "执行数学计算",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"expression": {"type": "string", "description": "数学表达式，如: 2 + 3 * 4"}
|
|
},
|
|
"required": ["expression"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**错误响应 (工具不存在):**
|
|
|
|
```json
|
|
{
|
|
"detail": "Tool not found"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 3. 健康检查
|
|
|
|
```http
|
|
GET /api/v1/tools/health
|
|
```
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"status": "healthy",
|
|
"timestamp": 1705315200.123,
|
|
"tools": ["search", "calculator", "weather", "translate", "knowledge", "code_interpreter"]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 自动测试 (Autotest)
|
|
|
|
### 4. 运行完整自动测试
|
|
|
|
```http
|
|
POST /api/v1/tools/autotest
|
|
```
|
|
|
|
**Query Parameters:**
|
|
|
|
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|
|
|------|------|------|--------|------|
|
|
| llm_model_id | string | 否 | - | LLM 模型ID |
|
|
| asr_model_id | string | 否 | - | ASR 模型ID |
|
|
| test_llm | boolean | 否 | true | 是否测试LLM |
|
|
| test_asr | boolean | 否 | true | 是否测试ASR |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"id": "abc12345",
|
|
"started_at": 1705315200.0,
|
|
"duration_ms": 2500,
|
|
"tests": [
|
|
{
|
|
"name": "Model Existence",
|
|
"passed": true,
|
|
"message": "Found model: GPT-4o",
|
|
"duration_ms": 15
|
|
},
|
|
{
|
|
"name": "API Connection",
|
|
"passed": true,
|
|
"message": "Latency: 150ms",
|
|
"duration_ms": 150
|
|
},
|
|
{
|
|
"name": "Temperature Setting",
|
|
"passed": true,
|
|
"message": "temperature=0.7"
|
|
},
|
|
{
|
|
"name": "Streaming Support",
|
|
"passed": true,
|
|
"message": "Received 15 chunks",
|
|
"duration_ms": 800
|
|
}
|
|
],
|
|
"summary": {
|
|
"passed": 4,
|
|
"failed": 0,
|
|
"total": 4
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 5. 测试单个 LLM 模型
|
|
|
|
```http
|
|
POST /api/v1/tools/autotest/llm/{model_id}
|
|
```
|
|
|
|
**Path Parameters:**
|
|
|
|
| 参数 | 类型 | 说明 |
|
|
|------|------|------|
|
|
| model_id | string | LLM 模型ID |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"id": "llm_test_001",
|
|
"started_at": 1705315200.0,
|
|
"duration_ms": 1200,
|
|
"tests": [
|
|
{
|
|
"name": "Model Existence",
|
|
"passed": true,
|
|
"message": "Found model: GPT-4o",
|
|
"duration_ms": 10
|
|
},
|
|
{
|
|
"name": "API Connection",
|
|
"passed": true,
|
|
"message": "Latency: 180ms",
|
|
"duration_ms": 180
|
|
},
|
|
{
|
|
"name": "Temperature Setting",
|
|
"passed": true,
|
|
"message": "temperature=0.7"
|
|
},
|
|
{
|
|
"name": "Streaming Support",
|
|
"passed": true,
|
|
"message": "Received 12 chunks",
|
|
"duration_ms": 650
|
|
}
|
|
],
|
|
"summary": {
|
|
"passed": 4,
|
|
"failed": 0,
|
|
"total": 4
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 6. 测试单个 ASR 模型
|
|
|
|
```http
|
|
POST /api/v1/tools/autotest/asr/{model_id}
|
|
```
|
|
|
|
**Path Parameters:**
|
|
|
|
| 参数 | 类型 | 说明 |
|
|
|------|------|------|
|
|
| model_id | string | ASR 模型ID |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"id": "asr_test_001",
|
|
"started_at": 1705315200.0,
|
|
"duration_ms": 800,
|
|
"tests": [
|
|
{
|
|
"name": "Model Existence",
|
|
"passed": true,
|
|
"message": "Found model: Whisper-1",
|
|
"duration_ms": 8
|
|
},
|
|
{
|
|
"name": "Hotwords Config",
|
|
"passed": true,
|
|
"message": "Hotwords: 3 words"
|
|
},
|
|
{
|
|
"name": "API Availability",
|
|
"passed": true,
|
|
"message": "Status: 200",
|
|
"duration_ms": 250
|
|
},
|
|
{
|
|
"name": "Language Config",
|
|
"passed": true,
|
|
"message": "Language: zh"
|
|
}
|
|
],
|
|
"summary": {
|
|
"passed": 4,
|
|
"failed": 0,
|
|
"total": 4
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 7. 发送测试消息
|
|
|
|
```http
|
|
POST /api/v1/tools/test-message
|
|
```
|
|
|
|
**Query Parameters:**
|
|
|
|
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|
|
|------|------|------|--------|------|
|
|
| llm_model_id | string | 是 | - | LLM 模型ID |
|
|
| message | string | 否 | "Hello, this is a test message." | 测试消息 |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"reply": "Hello! This is a test reply from GPT-4o.",
|
|
"usage": {
|
|
"prompt_tokens": 15,
|
|
"completion_tokens": 12,
|
|
"total_tokens": 27
|
|
}
|
|
}
|
|
```
|
|
|
|
**错误响应 (模型不存在):**
|
|
|
|
```json
|
|
{
|
|
"detail": "LLM Model not found"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 测试结果结构
|
|
|
|
### AutotestResult
|
|
|
|
```typescript
|
|
interface AutotestResult {
|
|
id: string; // 测试ID
|
|
started_at: number; // 开始时间戳
|
|
duration_ms: number; // 总耗时(毫秒)
|
|
tests: TestCase[]; // 测试用例列表
|
|
summary: TestSummary; // 测试摘要
|
|
}
|
|
|
|
interface TestCase {
|
|
name: string; // 测试名称
|
|
passed: boolean; // 是否通过
|
|
message: string; // 测试消息
|
|
duration_ms: number; // 耗时(毫秒)
|
|
}
|
|
|
|
interface TestSummary {
|
|
passed: number; // 通过数量
|
|
failed: number; // 失败数量
|
|
total: number; // 总数量
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 测试项目说明
|
|
|
|
### LLM 模型测试项目
|
|
|
|
| 测试名称 | 说明 |
|
|
|----------|------|
|
|
| Model Existence | 检查模型是否存在于数据库 |
|
|
| API Connection | 测试 API 连接并测量延迟 |
|
|
| Temperature Setting | 检查温度配置 |
|
|
| Streaming Support | 测试流式响应支持 |
|
|
|
|
### ASR 模型测试项目
|
|
|
|
| 测试名称 | 说明 |
|
|
|----------|------|
|
|
| Model Existence | 检查模型是否存在于数据库 |
|
|
| Hotwords Config | 检查热词配置 |
|
|
| API Availability | 测试 API 可用性 |
|
|
| Language Config | 检查语言配置 |
|
|
|
|
---
|
|
|
|
## 单元测试
|
|
|
|
项目包含完整的单元测试，位于 `api/tests/test_tools.py`。
|
|
|
|
### 测试用例概览
|
|
|
|
| 测试类 | 说明 |
|
|
|--------|------|
|
|
| TestToolsAPI | 工具列表、健康检查等基础功能测试 |
|
|
| TestAutotestAPI | 自动测试功能完整测试 |
|
|
|
|
### 运行测试
|
|
|
|
```bash
|
|
# 运行工具相关测试
|
|
pytest api/tests/test_tools.py -v
|
|
|
|
# 运行所有测试
|
|
pytest api/tests/ -v
|
|
```
|