Add vision model support and related configurations in Assistant

- Introduce new fields in AssistantConfig, schemas, and database models to support vision capabilities, including `vision_enabled` and `vision_model_resource_id`.
- Enhance validation logic in routes to ensure proper handling of vision models and their requirements.
- Update the AssistantPage and related frontend components to include options for enabling vision understanding and selecting appropriate vision models.
- Modify database seed scripts to include vision-related data for assistants, ensuring consistent setup.
- Refactor related functions to integrate vision model handling in the audio-visual processing pipeline.
This commit is contained in:
Xin Wang
2026-07-07 21:50:15 +08:00
parent c51a70e134
commit 5f71bf1681
10 changed files with 180 additions and 41 deletions

View File

@@ -128,6 +128,8 @@ export type Assistant = {
runtimeMode: RuntimeMode;
greeting: string;
enableInterrupt: boolean;
visionEnabled: boolean;
visionModelResourceId: string | null;
modelResourceIds: Partial<Record<ModelType, string>>;
knowledgeBaseId: string | null;
prompt: string;