Add support for image input in model resources and enhance related configurations
- Introduce a new `support_image_input` field in model resources, allowing models to indicate support for image input. - Update the backend models, schemas, and database seed scripts to accommodate the new field. - Enhance the AssistantConfig and related routes to handle image input capabilities, ensuring proper validation and error handling. - Modify the frontend components to include toggles for enabling visual understanding and filtering models based on image input support. - Implement necessary adjustments in the voice preview and pipeline to integrate video stream handling alongside audio functionalities.
This commit is contained in:
@@ -41,6 +41,7 @@ class AssistantConfig(BaseModel):
|
||||
tts_interface_type: str = "openai-tts"
|
||||
llm_values: dict = {}
|
||||
llm_secrets: dict = {}
|
||||
llm_support_image_input: bool = False
|
||||
stt_values: dict = {}
|
||||
stt_secrets: dict = {}
|
||||
tts_values: dict = {}
|
||||
@@ -80,3 +81,4 @@ class SignalingOffer(BaseModel):
|
||||
type: str
|
||||
assistant_id: str | None = None
|
||||
inline_config: AssistantConfig | None = None
|
||||
vision_enabled: bool = False
|
||||
|
||||
Reference in New Issue
Block a user