464 lines
9.0 KiB
Markdown
464 lines
9.0 KiB
Markdown
# LLM 模型 (LLM Model) API
|
||
|
||
LLM 模型 API 用于管理大语言模型的配置和调用。
|
||
|
||
## 基础信息
|
||
|
||
| 项目 | 值 |
|
||
|------|-----|
|
||
| Base URL | `/api/v1/llm` |
|
||
| 认证方式 | Bearer Token (预留) |
|
||
|
||
---
|
||
|
||
## 数据模型
|
||
|
||
### LLMModel
|
||
|
||
```typescript
|
||
interface LLMModel {
|
||
id: string; // 模型唯一标识 (8位UUID)
|
||
user_id: number; // 所属用户ID
|
||
name: string; // 模型显示名称
|
||
vendor: string; // 供应商: "OpenAI Compatible" | "Dify" | "FastGPT" | 等
|
||
type: string; // 类型: "text" | "embedding" | "rerank"
|
||
base_url: string; // API Base URL
|
||
api_key: string; // API Key
|
||
model_name?: string; // 实际模型名称，如 "gpt-4o"
|
||
temperature?: number; // 温度参数 (0-2)
|
||
context_length?: int; // 上下文长度
|
||
enabled: boolean; // 是否启用
|
||
created_at: string;
|
||
updated_at: string;
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## API 端点
|
||
|
||
### 1. 获取 LLM 模型列表
|
||
|
||
```http
|
||
GET /api/v1/llm
|
||
```
|
||
|
||
**Query Parameters:**
|
||
|
||
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|
||
|------|------|------|--------|------|
|
||
| model_type | string | 否 | - | 过滤类型: "text" \| "embedding" \| "rerank" |
|
||
| enabled | boolean | 否 | - | 过滤启用状态 |
|
||
| page | int | 否 | 1 | 页码 |
|
||
| limit | int | 否 | 50 | 每页数量 |
|
||
|
||
**Response:**
|
||
|
||
```json
|
||
{
|
||
"total": 5,
|
||
"page": 1,
|
||
"limit": 50,
|
||
"list": [
|
||
{
|
||
"id": "abc12345",
|
||
"user_id": 1,
|
||
"name": "GPT-4o",
|
||
"vendor": "OpenAI Compatible",
|
||
"type": "text",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-***",
|
||
"model_name": "gpt-4o",
|
||
"temperature": 0.7,
|
||
"context_length": 128000,
|
||
"enabled": true,
|
||
"created_at": "2024-01-15T10:30:00Z",
|
||
"updated_at": "2024-01-15T10:30:00Z"
|
||
},
|
||
{
|
||
"id": "def67890",
|
||
"user_id": 1,
|
||
"name": "Embedding-3-Small",
|
||
"vendor": "OpenAI Compatible",
|
||
"type": "embedding",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-***",
|
||
"model_name": "text-embedding-3-small",
|
||
"enabled": true
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 2. 获取单个 LLM 模型详情
|
||
|
||
```http
|
||
GET /api/v1/llm/{id}
|
||
```
|
||
|
||
**Path Parameters:**
|
||
|
||
| 参数 | 类型 | 说明 |
|
||
|------|------|------|
|
||
| id | string | 模型ID |
|
||
|
||
**Response:**
|
||
|
||
```json
|
||
{
|
||
"id": "abc12345",
|
||
"user_id": 1,
|
||
"name": "GPT-4o",
|
||
"vendor": "OpenAI Compatible",
|
||
"type": "text",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-***",
|
||
"model_name": "gpt-4o",
|
||
"temperature": 0.7,
|
||
"context_length": 128000,
|
||
"enabled": true,
|
||
"created_at": "2024-01-15T10:30:00Z",
|
||
"updated_at": "2024-01-15T10:30:00Z"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 3. 创建 LLM 模型
|
||
|
||
```http
|
||
POST /api/v1/llm
|
||
```
|
||
|
||
**Request Body:**
|
||
|
||
```json
|
||
{
|
||
"name": "GPT-4o",
|
||
"vendor": "OpenAI Compatible",
|
||
"type": "text",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-your-api-key",
|
||
"model_name": "gpt-4o",
|
||
"temperature": 0.7,
|
||
"context_length": 128000,
|
||
"enabled": true
|
||
}
|
||
```
|
||
|
||
**Fields 说明:**
|
||
|
||
| 字段 | 类型 | 必填 | 说明 |
|
||
|------|------|------|------|
|
||
| name | string | 是 | 模型显示名称 |
|
||
| vendor | string | 是 | 供应商名称 |
|
||
| type | string | 是 | 模型类型: "text" / "embedding" / "rerank" |
|
||
| base_url | string | 是 | API Base URL |
|
||
| api_key | string | 是 | API Key |
|
||
| model_name | string | 否 | 实际模型名称 |
|
||
| temperature | number | 否 | 温度参数，默认 0.7 |
|
||
| context_length | int | 否 | 上下文长度 |
|
||
| enabled | boolean | 否 | 是否启用，默认 true |
|
||
| id | string | 否 | 指定模型ID，默认自动生成 |
|
||
|
||
---
|
||
|
||
### 4. 更新 LLM 模型
|
||
|
||
```http
|
||
PUT /api/v1/llm/{id}
|
||
```
|
||
|
||
**Request Body:** (部分更新)
|
||
|
||
```json
|
||
{
|
||
"name": "GPT-4o-Updated",
|
||
"temperature": 0.8,
|
||
"enabled": false
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 5. 删除 LLM 模型
|
||
|
||
```http
|
||
DELETE /api/v1/llm/{id}
|
||
```
|
||
|
||
**Response:**
|
||
|
||
```json
|
||
{
|
||
"message": "Deleted successfully"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 6. 测试 LLM 模型连接
|
||
|
||
```http
|
||
POST /api/v1/llm/{id}/test
|
||
```
|
||
|
||
**Response:**
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"latency_ms": 150,
|
||
"message": "Connection successful"
|
||
}
|
||
```
|
||
|
||
**错误响应:**
|
||
|
||
```json
|
||
{
|
||
"success": false,
|
||
"latency_ms": 200,
|
||
"message": "HTTP Error: 401 - Unauthorized"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 7. 与 LLM 模型对话
|
||
|
||
```http
|
||
POST /api/v1/llm/{id}/chat
|
||
```
|
||
|
||
**Query Parameters:**
|
||
|
||
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|
||
|------|------|------|--------|------|
|
||
| message | string | 是 | - | 用户消息 |
|
||
| system_prompt | string | 否 | - | 系统提示词 |
|
||
| max_tokens | int | 否 | 1000 | 最大生成token数 |
|
||
| temperature | number | 否 | 模型配置 | 温度参数 |
|
||
|
||
**Response:**
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"reply": "您好！有什么可以帮助您的？",
|
||
"usage": {
|
||
"prompt_tokens": 20,
|
||
"completion_tokens": 15,
|
||
"total_tokens": 35
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 8. 预览模型输出
|
||
|
||
```http
|
||
POST /api/v1/llm/{id}/preview
|
||
```
|
||
|
||
预览模型输出，支持 text(chat) 与 embedding 两类模型。
|
||
|
||
**Request Body:**
|
||
|
||
```json
|
||
{
|
||
"message": "请介绍一下你自己",
|
||
"system_prompt": "你是一个专业的AI助手",
|
||
"max_tokens": 512,
|
||
"temperature": 0.7
|
||
}
|
||
```
|
||
|
||
**Response (text model):**
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"reply": "您好！我是一个...",
|
||
"usage": {
|
||
"prompt_tokens": 20,
|
||
"completion_tokens": 50,
|
||
"total_tokens": 70
|
||
},
|
||
"latency_ms": 1500,
|
||
"error": null
|
||
}
|
||
```
|
||
|
||
**Response (embedding model):**
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"reply": "Embedding generated successfully. dims=1536. head=[0.012345, -0.023456, ...]",
|
||
"usage": {
|
||
"prompt_tokens": 10,
|
||
"total_tokens": 10
|
||
},
|
||
"latency_ms": 800,
|
||
"error": null
|
||
}
|
||
```
|
||
|
||
**Fields 说明:**
|
||
|
||
| 字段 | 类型 | 必填 | 说明 |
|
||
|------|------|------|------|
|
||
| message | string | 是 | 用户消息/嵌入文本 |
|
||
| system_prompt | string | 否 | 系统提示词 (仅 text 模型) |
|
||
| max_tokens | int | 否 | 最大生成 token 数 (默认 512) |
|
||
| temperature | float | 否 | 温度参数 |
|
||
| api_key | string | 否 | 覆盖模型配置的 API Key |
|
||
|
||
---
|
||
|
||
## Schema 定义
|
||
|
||
```python
|
||
from enum import Enum
|
||
from pydantic import BaseModel
|
||
from typing import Optional
|
||
from datetime import datetime
|
||
|
||
class LLMModelType(str, Enum):
|
||
TEXT = "text"
|
||
EMBEDDING = "embedding"
|
||
RERANK = "rerank"
|
||
|
||
class LLMModelBase(BaseModel):
|
||
name: str
|
||
vendor: str
|
||
type: LLMModelType
|
||
base_url: str
|
||
api_key: str
|
||
model_name: Optional[str] = None
|
||
temperature: Optional[float] = None
|
||
context_length: Optional[int] = None
|
||
enabled: bool = True
|
||
|
||
class LLMModelCreate(LLMModelBase):
|
||
id: Optional[str] = None
|
||
|
||
class LLMModelUpdate(BaseModel):
|
||
name: Optional[str] = None
|
||
vendor: Optional[str] = None
|
||
base_url: Optional[str] = None
|
||
api_key: Optional[str] = None
|
||
model_name: Optional[str] = None
|
||
temperature: Optional[float] = None
|
||
context_length: Optional[int] = None
|
||
enabled: Optional[bool] = None
|
||
|
||
class LLMModelOut(LLMModelBase):
|
||
id: str
|
||
user_id: int
|
||
created_at: datetime
|
||
updated_at: datetime
|
||
|
||
class Config:
|
||
from_attributes = True
|
||
|
||
class LLMModelTestResponse(BaseModel):
|
||
success: bool
|
||
latency_ms: int
|
||
message: str
|
||
```
|
||
|
||
---
|
||
|
||
## 供应商配置示例
|
||
|
||
### OpenAI Compatible (OpenAI Endpoint)
|
||
|
||
```json
|
||
{
|
||
"vendor": "OpenAI Compatible",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-xxx",
|
||
"model_name": "gpt-4o",
|
||
"type": "text",
|
||
"temperature": 0.7
|
||
}
|
||
```
|
||
|
||
### OpenAI Compatible
|
||
|
||
```json
|
||
{
|
||
"vendor": "OpenAI Compatible",
|
||
"base_url": "https://api.siliconflow.com/v1",
|
||
"api_key": "sf-xxx",
|
||
"model_name": "deepseek-v3",
|
||
"type": "text",
|
||
"temperature": 0.7
|
||
}
|
||
```
|
||
|
||
### Dify
|
||
|
||
```json
|
||
{
|
||
"vendor": "Dify",
|
||
"base_url": "https://your-dify.domain.com/v1",
|
||
"api_key": "app-xxx",
|
||
"model_name": "gpt-4",
|
||
"type": "text"
|
||
}
|
||
```
|
||
|
||
### Embedding 模型
|
||
|
||
```json
|
||
{
|
||
"vendor": "OpenAI Compatible",
|
||
"base_url": "https://api.openai.com/v1",
|
||
"api_key": "sk-xxx",
|
||
"model_name": "text-embedding-3-small",
|
||
"type": "embedding"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 单元测试
|
||
|
||
项目包含完整的单元测试，位于 `api/tests/test_llm.py`。
|
||
|
||
### 测试用例概览
|
||
|
||
| 测试方法 | 说明 |
|
||
|----------|------|
|
||
| test_get_llm_models_empty | 空数据库获取测试 |
|
||
| test_create_llm_model | 创建模型测试 |
|
||
| test_create_llm_model_minimal | 最小数据创建测试 |
|
||
| test_get_llm_model_by_id | 获取单个模型测试 |
|
||
| test_get_llm_model_not_found | 获取不存在模型测试 |
|
||
| test_update_llm_model | 更新模型测试 |
|
||
| test_delete_llm_model | 删除模型测试 |
|
||
| test_list_llm_models_with_pagination | 分页测试 |
|
||
| test_filter_llm_models_by_type | 按类型过滤测试 |
|
||
| test_filter_llm_models_by_enabled | 按启用状态过滤测试 |
|
||
| test_create_llm_model_with_all_fields | 全字段创建测试 |
|
||
| test_test_llm_model_success | 测试连接成功测试 |
|
||
| test_test_llm_model_failure | 测试连接失败测试 |
|
||
| test_different_llm_vendors | 多供应商测试 |
|
||
| test_embedding_llm_model | Embedding 模型测试 |
|
||
|
||
### 运行测试
|
||
|
||
```bash
|
||
# 运行 LLM 相关测试
|
||
pytest api/tests/test_llm.py -v
|
||
|
||
# 运行所有测试
|
||
pytest api/tests/ -v
|
||
```
|