Add vision model support and related configurations in Assistant
- Introduce new fields in AssistantConfig, schemas, and database models to support vision capabilities, including `vision_enabled` and `vision_model_resource_id`. - Enhance validation logic in routes to ensure proper handling of vision models and their requirements. - Update the AssistantPage and related frontend components to include options for enabling vision understanding and selecting appropriate vision models. - Modify database seed scripts to include vision-related data for assistants, ensuring consistent setup. - Refactor related functions to integrate vision model handling in the audio-visual processing pipeline.
This commit is contained in:
@@ -42,6 +42,15 @@ class AssistantConfig(BaseModel):
|
||||
llm_values: dict = {}
|
||||
llm_secrets: dict = {}
|
||||
llm_support_image_input: bool = False
|
||||
vision_enabled: bool = False
|
||||
vision_model_resource_id: str | None = None
|
||||
vision_model: str = ""
|
||||
vision_llm_interface_type: str = "openai-llm"
|
||||
vision_llm_values: dict = {}
|
||||
vision_llm_secrets: dict = {}
|
||||
vision_llm_support_image_input: bool = False
|
||||
vision_llm_api_key: str = ""
|
||||
vision_llm_base_url: str = ""
|
||||
stt_values: dict = {}
|
||||
stt_secrets: dict = {}
|
||||
tts_values: dict = {}
|
||||
|
||||
Reference in New Issue
Block a user