Commit Graph

14 Commits

Author SHA1 Message Date
Xin Wang
e25dfd4003 Add support for Xfyun ASR and TTS services in the backend
- Introduce new Xfyun ASR and TTS services, enabling integration with iFlytek's voice recognition and synthesis capabilities.
- Update AssistantConfig model to include interface types for STT and TTS.
- Enhance credential testing to validate Xfyun credentials.
- Modify service factory to create Xfyun services based on configuration.
- Update README with new configuration details for Xfyun integration.
- Add new frontend components for visualizing audio streams and managing user interactions.
2026-06-11 10:51:08 +08:00
Xin Wang
2c2af1f2cd Enhance voice interaction and transcript handling in the assistant
- Add a new Docker configuration for the UI in launch.json to facilitate development.
- Refactor pipeline.py to integrate a TranscriptProcessor for managing user and assistant transcripts, including event handlers for real-time updates and message handling.
- Update useVoicePreview.ts to establish a data channel for sending and receiving text messages, improving interaction flow.
- Modify AssistantPage.tsx to support displaying chat messages and sending user input, enhancing the user experience during voice interactions.
- Revise DebugTranscriptPanel to dynamically render chat messages with timestamps, improving the visual representation of conversation history.
2026-06-10 15:11:34 +08:00
Xin Wang
0adb3ed8a1 Add initial setup for local HTTPS debugging and Nginx configuration
- Introduce `setup-certs.sh` script for generating trusted local TLS certificates using mkcert.
- Add Nginx configuration files for local and Docker environments to handle HTTPS requests and proxy to backend services.
- Update `docker-compose.yaml` to include Nginx service for unified TLS entry and adjust frontend service ports for local development.
- Create `AGENTS.md` and `README.md` files to document the local HTTPS setup process and usage instructions.
- Modify backend startup commands in `README.md` for consistency with new requirements.
- Add `.gitignore` to exclude generated certificates from version control.
2026-06-10 13:37:24 +08:00
Xin Wang
e94d98e947 fix frontend voice preview fallback 2026-06-10 12:36:18 +08:00
Xin Wang
4a948ee609 fix backend TTS provider compatibility 2026-06-10 12:32:55 +08:00
Xin Wang
ac3f4dd806 Enhance voice interaction features and introduce voice preview functionality
- Update README to reflect the integration of the DebugVoicePanel with WebSocket support for voice interactions.
- Refactor voice_webrtc.py to improve error handling during WebRTC signaling and include assistant_id in the offer payload.
- Add useVoicePreview hook to manage microphone access and WebRTC connections for real-time voice previews.
- Modify AssistantPage to incorporate new visualizer options and pass assistantId to DebugVoicePanel, enhancing user experience during audio interactions.
- Update API model to include new fields for voice, speed, and language, supporting TTS and ASR configurations.
2026-06-10 10:17:46 +08:00
Xin Wang
c64b7dcf99 Enhance credential management and testing functionality
- Introduce new fields for voice, speed, and language in the AssistantConfig and ProviderCredential models to support TTS and ASR configurations.
- Update the database schema and seeding script to accommodate the new fields, ensuring backward compatibility.
- Implement credential testing endpoints and logic to validate OpenAI-compatible credentials, enhancing user experience and reliability.
- Modify frontend components to include new fields in the credential forms and improve connection testing feedback.
- Refactor related services and API interactions to support the new credential testing feature.
2026-06-09 14:42:25 +08:00
Xin Wang
3661dab81c reorder seed data 2026-06-09 13:38:45 +08:00
Xin Wang
6acbac7d3b Update seed_credentials.sql with new model names and API keys
- Change 'DeepSeek-V3' to 'DeepSeek-Chat' and update its API key.
- Rename 'OpenAI TTS' to 'SiliconFlow-CosyVoice2-0.5B' and update its details.
- Add new models: 'SiliconFlow-TeleSpeechASR' and 'SiliconFlow-Qwen3-Embedding-4B' with corresponding API keys and configurations.
- Adjust existing entries to ensure consistency in the database seeding process.
2026-06-09 11:31:49 +08:00
Xin Wang
519cc0fefe Refactor assistant configuration and database seeding
- Update Makefile to include new database seed commands for assistants and credentials.
- Refactor assistant model to use explicit fields instead of a config dictionary, improving data integrity and clarity.
- Implement new seeding SQL script for assistants, ensuring dependencies on credentials are respected.
- Modify backend routes and frontend components to accommodate the new assistant structure, including direct field access for prompt, API URL, and keys.
- Enhance the AssistantPage component to handle the new data structure and streamline the save process for different assistant types.
2026-06-09 10:37:29 +08:00
Xin Wang
30b96bb3be Add duplicate functionality for assistants and credentials
Implement server-side duplication for both assistants and credentials, allowing users to create copies with unique IDs and modified names. Update the respective API routes and frontend components to handle duplication requests, ensuring sensitive information is securely managed. Enhance the AssistantPage and ComponentsModelsPage to support this new feature, including loading and error handling for the duplication process.
2026-06-09 09:48:43 +08:00
Xin Wang
b444ea777c Implement knowledge base management and enhance assistant configuration
Add CRUD functionality for knowledge bases, including routes for listing, creating, updating, and deleting knowledge bases. Update the assistant model to include foreign key references to knowledge bases and modify the assistant configuration to handle external API keys securely. Refactor related services and routes to accommodate these changes, ensuring proper handling of credential resolution and configuration normalization.
2026-06-09 08:31:39 +08:00
Xin Wang
7e8e8624b4 Enhance AI Video Assistant platform with new Makefile for development commands, update CORS origins for local access, and implement API client for credential management. Add seed data for model credentials and refactor ComponentsModelsPage to utilize API for dynamic data loading. Update Next.js configuration for Turbopack compatibility. 2026-06-08 22:39:45 +08:00
Xin Wang
42cab2a6ef Initial commit: AI Video Assistant fullstack platform.
Add pipecat-based backend with WebRTC/WS voice routes, Next.js frontend, and Docker Compose orchestration.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-08 13:51:28 +08:00