Add support for image input in model resources and enhance related configurations
- Introduce a new `support_image_input` field in model resources, allowing models to indicate support for image input. - Update the backend models, schemas, and database seed scripts to accommodate the new field. - Enhance the AssistantConfig and related routes to handle image input capabilities, ensuring proper validation and error handling. - Modify the frontend components to include toggles for enabling visual understanding and filtering models based on image input support. - Implement necessary adjustments in the voice preview and pipeline to integrate video stream handling alongside audio functionalities.
This commit is contained in:
@@ -86,6 +86,9 @@ async def resolve_runtime_config(
|
||||
realtimeModel=str(_value(realtime_resource, "modelId", "")),
|
||||
llm_values=(llm_resource.values or {}) if llm_resource else {},
|
||||
llm_secrets=(llm_resource.secrets or {}) if llm_resource else {},
|
||||
llm_support_image_input=(
|
||||
bool(llm_resource.support_image_input) if llm_resource else False
|
||||
),
|
||||
stt_values=(stt_resource.values or {}) if stt_resource else {},
|
||||
stt_secrets=(stt_resource.secrets or {}) if stt_resource else {},
|
||||
tts_values=(tts_resource.values or {}) if tts_resource else {},
|
||||
|
||||
Reference in New Issue
Block a user