Commit Graph

16 Commits

Author SHA1 Message Date
Xin Wang
35d24acf40 Refactor backend configuration management and update environment settings
- Replace the `config.py` module with a new `settings.py` to streamline environment variable management, focusing on database, CORS, and TURN settings.
- Update references throughout the backend codebase to use the new `settings` module instead of the deprecated `config`.
- Modify the `.env.example` file to reflect the new configuration approach, indicating that model provider credentials should be maintained separately.
- Enhance the `AssistantConfig` model to clarify the source of runtime connection information, ensuring it is injected from model resources rather than relying on defaults from the environment.
- Introduce new user scripts for audio and video management in the Tampermonkey environment, enhancing WebRTC capabilities.
2026-07-09 21:40:29 +08:00
Xin Wang
03b532dd09 Implement Alembic for database migrations and update FastAPI initialization
- Integrate Alembic for managing database schema migrations, replacing the previous SQL schema management.
- Update the FastAPI application to synchronize interface definitions at startup.
- Modify the Docker Compose command to run Alembic migrations before starting the API.
- Enhance the Makefile with new commands for database migration and revision management.
- Remove outdated SQL schema and seed files, transitioning to a more dynamic migration approach.
- Add initial migration scripts and configuration for Alembic, ensuring a structured database evolution.
2026-07-09 16:29:10 +08:00
Xin Wang
54329aa516 Add Agent capabilities and configurations across the application
- Extend the AssistantConfig model to include agent_interface_type, agent_values, and agent_secrets for enhanced agent management.
- Update ModelType to include "Agent" for broader capability support.
- Seed new agent models and configurations in the database for Dify, FastGPT, and OpenCode applications.
- Modify the Assistant routes to validate and handle agent capabilities.
- Implement agent resource resolution in the config resolver for runtime configuration.
- Enhance the interface catalog with agent-specific fields and definitions.
- Update frontend components to manage agent configurations, including form handling and state management for agent-related inputs.
2026-07-09 15:29:25 +08:00
Xin Wang
2fb7399953 Enhance Makefile and database schema for improved management
- Add new database management targets: db-up, db-schema, db-seed-interface-definitions, and db-init to streamline database setup and seeding processes.
- Update db-seed and db-reset dependencies to include new initialization steps for better data integrity.
- Introduce new SQL files for schema definition and interface definitions seeding, ensuring a consistent database structure.
- Refactor existing seed scripts to align with new dependencies and improve clarity in database operations.
2026-07-07 22:08:42 +08:00
Xin Wang
5f71bf1681 Add vision model support and related configurations in Assistant
- Introduce new fields in AssistantConfig, schemas, and database models to support vision capabilities, including `vision_enabled` and `vision_model_resource_id`.
- Enhance validation logic in routes to ensure proper handling of vision models and their requirements.
- Update the AssistantPage and related frontend components to include options for enabling vision understanding and selecting appropriate vision models.
- Modify database seed scripts to include vision-related data for assistants, ensuring consistent setup.
- Refactor related functions to integrate vision model handling in the audio-visual processing pipeline.
2026-07-07 21:50:15 +08:00
Xin Wang
c5db918830 Add support for image input in model resources and enhance related configurations
- Introduce a new `support_image_input` field in model resources, allowing models to indicate support for image input.
- Update the backend models, schemas, and database seed scripts to accommodate the new field.
- Enhance the AssistantConfig and related routes to handle image input capabilities, ensuring proper validation and error handling.
- Modify the frontend components to include toggles for enabling visual understanding and filtering models based on image input support.
- Implement necessary adjustments in the voice preview and pipeline to integrate video stream handling alongside audio functionalities.
2026-07-07 19:41:45 +08:00
Xin Wang
0309c154b5 Implement StepFun Realtime service and enhance AssistantConfig
- Add new fields to AssistantConfig for realtime interface configuration, including types, values, and secrets.
- Introduce StepFunRealtimeService to handle speech-to-speech processing via WebSocket, integrating STT, LLM, and TTS functionalities.
- Refactor pipeline execution to support a new realtime mode, allowing direct text input processing and immediate responses.
- Update model resource testing to include validation for StepFun Realtime connections.
- Enhance service factory to create realtime services based on configuration settings.
- Modify README documentation to reflect new realtime capabilities and usage instructions.
2026-06-14 23:41:40 +08:00
Xin Wang
90e3e8a0c0 Refactor backend to support interface-definition driven model resources
- Introduce a new model structure for managing interface definitions and model resources, enhancing the backend's capability to handle various service integrations.
- Update the Makefile to reflect changes in database seeding and resource management commands.
- Remove the deprecated credentials management routes and replace them with a unified model registry API.
- Modify existing routes and schemas to align with the new model structure, ensuring seamless integration with the frontend.
- Enhance database seeding scripts to populate new model resources and their configurations.
- Update README documentation to reflect the new architecture and usage instructions for model resources and interface definitions.
2026-06-14 19:36:12 +08:00
Xin Wang
e25dfd4003 Add support for Xfyun ASR and TTS services in the backend
- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities.
- Update AssistantConfig model to include interface types for STT and TTS.
- Enhance credential testing to validate Xfyun credentials.
- Modify service factory to create Xfyun services based on configuration.
- Update README with new configuration details for Xfyun integration.
- Add new frontend components for visualizing audio streams and managing user interactions.
2026-06-11 10:51:08 +08:00
Xin Wang
c64b7dcf99 Enhance credential management and testing functionality
- Introduce new fields for voice, speed, and language in the AssistantConfig and ProviderCredential models to support TTS and ASR configurations.
- Update the database schema and seeding script to accommodate the new fields, ensuring backward compatibility.
- Implement credential testing endpoints and logic to validate OpenAI-compatible credentials, enhancing user experience and reliability.
- Modify frontend components to include new fields in the credential forms and improve connection testing feedback.
- Refactor related services and API interactions to support the new credential testing feature.
2026-06-09 14:42:25 +08:00
Xin Wang
3661dab81c reorder seed data 2026-06-09 13:38:45 +08:00
Xin Wang
6acbac7d3b Update seed_credentials.sql with new model names and API keys
- Change 'DeepSeek-V3' to 'DeepSeek-Chat' and update its API key.
- Rename 'OpenAI TTS' to 'SiliconFlow-CosyVoice2-0.5B' and update its details.
- Add new models: 'SiliconFlow-TeleSpeechASR' and 'SiliconFlow-Qwen3-Embedding-4B' with corresponding API keys and configurations.
- Adjust existing entries to ensure consistency in the database seeding process.
2026-06-09 11:31:49 +08:00
Xin Wang
519cc0fefe Refactor assistant configuration and database seeding
- Update Makefile to include new database seed commands for assistants and credentials.
- Refactor assistant model to use explicit fields instead of a config dictionary, improving data integrity and clarity.
- Implement new seeding SQL script for assistants, ensuring dependencies on credentials are respected.
- Modify backend routes and frontend components to accommodate the new assistant structure, including direct field access for prompt, API URL, and keys.
- Enhance the AssistantPage component to handle the new data structure and streamline the save process for different assistant types.
2026-06-09 10:37:29 +08:00
Xin Wang
b444ea777c Implement knowledge base management and enhance assistant configuration
Add CRUD functionality for knowledge bases, including routes for listing, creating, updating, and deleting knowledge bases. Update the assistant model to include foreign key references to knowledge bases and modify the assistant configuration to handle external API keys securely. Refactor related services and routes to accommodate these changes, ensuring proper handling of credential resolution and configuration normalization.
2026-06-09 08:31:39 +08:00
Xin Wang
7e8e8624b4 Enhance AI Video Assistant platform with new Makefile for development commands, update CORS origins for local access, and implement API client for credential management. Add seed data for model credentials and refactor ComponentsModelsPage to utilize API for dynamic data loading. Update Next.js configuration for Turbopack compatibility. 2026-06-08 22:39:45 +08:00
Xin Wang
42cab2a6ef Initial commit: AI Video Assistant fullstack platform.
Add pipecat-based backend with WebRTC/WS voice routes, Next.js frontend, and Docker Compose orchestration.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-08 13:51:28 +08:00