f3612a710d
Add fastgpt as seperate assistant mode
Xin Wang
2026-03-11 08:37:34 +08:00
13684d498b
feat/fix(frontend): update shadcn compnents, fix debug drawer layout and font sizes
Xin Wang
2026-03-10 16:21:58 +08:00
47293ac46d
feat: Add core UI components, Assistants page, Dashscope and Volcengine agent configurations, and a WAV client example.
Xin Wang
2026-03-10 03:31:39 +08:00
373be4eb97
feat: Add DashScope and Volcengine agent configurations, a WAV client for duplex testing, and an Assistants UI page.
Xin Wang
2026-03-10 03:13:47 +08:00
e4ccec6cc1
feat: Introduce DashScope agent configuration, a WAV client for duplex testing, and new UI components for assistants.
Xin Wang
2026-03-10 02:25:52 +08:00
b300b469dc
Update documentation for Realtime Agent Studio with enhanced content and structure
Xin Wang
2026-03-09 05:38:43 +08:00
e41d34fe23
Add DashScope agent configuration files for VAD, LLM, TTS, and ASR services
Xin Wang
2026-03-08 23:28:08 +08:00
aeeeee20d1
Add Volcengine support for TTS and ASR services
Xin Wang
2026-03-08 23:09:50 +08:00
3604db21eb
Remove obsolete audio example files from the project
Xin Wang
2026-03-06 14:43:11 +08:00
65ae2287d5
Update documentation for assistant configuration and interaction models
Xin Wang
2026-03-06 14:38:59 +08:00
da38157638
Add ASR interim results support in Assistant model and API
Xin Wang
2026-03-06 12:58:54 +08:00
e11c3abb9e
Implement DashScope ASR provider and enhance ASR service architecture
Xin Wang
2026-03-06 11:44:39 +08:00
7e0b777923
Refactor project structure and enhance backend integration
Xin Wang
2026-03-06 09:51:56 +08:00
4e2450e800
Refactor backend integration and service architecture
Xin Wang
2026-03-06 09:00:43 +08:00
6b589a1b7c
Enhance session management and logging configuration
Xin Wang
2026-03-05 21:44:23 +08:00
1cecbaa172
Update .gitignore and add audio example file
Xin Wang
2026-03-05 21:28:17 +08:00
935f2fbd1f
Refactor assistant configuration management and update documentation
Xin Wang
2026-03-05 21:24:15 +08:00
d0a6419990
Remove duplicate entry for Vocode Core from the roadmap documentation, streamlining the list of reference projects.
Xin Wang
2026-03-05 13:22:21 +08:00
d0897aca92
Update documentation to reflect rebranding from AI Video Assistant to Realtime Agent Studio (RAS)
Xin Wang
2026-03-02 15:11:33 +08:00
70b4043f9b
Enhance DebugDrawer to support voice prompts in text prompt dialogs
Xin Wang
2026-03-02 15:10:03 +08:00
3aa9e0f432
Enhance DuplexPipeline to support follow-up context for manual opener tool calls
Xin Wang
2026-03-02 14:27:44 +08:00
fb017f9952
Refactor selectedToolSchemas logic in DebugDrawer to simplify tool ID normalization. Removed redundant inclusion of DEBUG_CLIENT_TOOLS, enhancing code clarity and performance.
Xin Wang
2026-03-02 12:40:00 +08:00
00b88c5afa
Add manual opener tool calls to Assistant model and API
Xin Wang
2026-03-02 12:34:42 +08:00
b5cdb76e52
Implement initial generated opener logic in DuplexPipeline to utilize tool-capable assistant turns when tools are available. Update tests to verify the correct behavior of the generated opener under various conditions, ensuring proper handling of user input and task management.
Xin Wang
2026-03-02 02:47:30 +08:00
4d553de34d
Refactor assistant greeting logic to conditionally use system prompt for generated openers. Update related tests to verify new behavior and ensure correct metadata handling in API responses. Enhance UI to reflect changes in opener management based on generated opener settings.
Xin Wang
2026-03-02 02:38:45 +08:00
31b3969b96
Enhance ToolLibrary by adding sourceKey to ToolParameterDraft and updating related functions for improved schema management. Introduce normalization functions for object schemas and defaults, and refactor buildToolParameterConfig to utilize these enhancements. Update state management in ToolLibraryPage to accommodate new schema handling and defaults integration.
Xin Wang
2026-03-02 02:18:28 +08:00
531688aa6b
Enhance API documentation by adding new endpoints for ASR preview, assistant configuration retrieval, and knowledge base management. Update existing assistant and tool definitions for improved clarity and functionality. Remove outdated sections from history records documentation, ensuring a streamlined reference for users.
Xin Wang
2026-03-02 01:56:38 +08:00
3626297211
Implement schema editor functionality in ToolLibrary, allowing users to manage tool parameters with JSON schema validation. Add a drawer for schema editing, enhance state management for schema-related errors, and integrate schema defaults into tool parameter configuration. Update UI to include a button for opening the schema drawer.
Xin Wang
2026-03-02 01:54:54 +08:00
1561056a3d
Add voice_choice_prompt and text_choice_prompt tools to API and UI. Implement state management and parameter definitions for user selection prompts, enhancing user interaction and experience.
Xin Wang
2026-03-02 00:49:31 +08:00
3a5d27d6c3
Implement runtime configuration debugging in DebugDrawer by adding a new function to format session metadata and WebSocket configuration. Update the display logic to enhance clarity and user experience, including renaming UI elements for better context.
Xin Wang
2026-03-01 23:14:08 +08:00
3643431565
Enhance WebSocket session configuration by introducing an optional config.resolved event, which provides a public snapshot of the session's configuration. Update the API reference documentation to clarify the conditions under which this event is emitted and the details it includes. Modify session management to respect the new setting for emitting configuration details, ensuring sensitive information remains secure. Update tests to validate the new behavior and ensure compliance with the updated configuration schema.
Xin Wang
2026-03-01 23:08:44 +08:00
2418df80e5
Revamp documentation structure in mkdocs.yml by reorganizing navigation for improved accessibility. Remove outdated content from previous sections and introduce new topics including detailed guides on assistant management, configuration options, and tool integrations. Enhance API reference documentation with comprehensive error codes and WebSocket protocol details. Add new sections for automated testing, data analysis, and knowledge base management, ensuring a cohesive and user-friendly documentation experience.
Xin Wang
2026-03-01 22:38:50 +08:00
6a46ec69f4
Enhance WebSocket session management by requiring assistant_id as a query parameter for connection. Update API reference documentation to reflect changes in message flow and metadata validation rules, including the introduction of whitelists for allowed metadata fields and restrictions on sensitive keys. Refactor client examples to align with the new session initiation process.
Xin Wang
2026-03-01 14:10:38 +08:00
b4fa664d73
Refactor WebSocket authentication handling by removing auth requirements from the hello message. Update related documentation and schemas to reflect the changes in authentication strategy, simplifying the connection process.
Xin Wang
2026-02-28 17:33:40 +08:00
0821d73e7c
Add API reference documentation for WebSocket communication. Update mkdocs.yml to include new API reference section.
Xin Wang
2026-02-28 14:37:58 +08:00
aae41d4512
Clear stale ASR capture on end of utterance in DuplexPipeline. Add test to verify behavior when conversation state changes, ensuring proper handling of ASR capture variables.
Xin Wang
2026-02-28 12:32:35 +08:00
c4c473105e
Add start-dev.ps1 script to automate the launch of development services in the pycall conda environment. The script initiates the API, Web, and Engine services in separate PowerShell windows, enhancing the development workflow.
Xin Wang
2026-02-28 11:26:52 +08:00
8b59569b99
Add choice_prompt tool to Assistants and DebugDrawer. Implement state management for choice prompts, including normalization of options and handling user selections. Enhance UI interactions for improved user experience.
Xin Wang
2026-02-28 10:39:33 +08:00
e40899613f
Refactor DebugDrawer to use a ref for text prompt dialog state management. Update closeTextPromptDialog function to utilize the ref for improved handling of dialog state, ensuring consistent behavior during interactions.
Xin Wang
2026-02-27 18:22:45 +08:00
915d2f4bd8
Refactor DebugDrawer to utilize a ref for client tool state management. Introduce isClientToolEnabledLive function for real-time tool state checks, ensuring accurate handling of tool enablement in debug settings.
Xin Wang
2026-02-27 18:10:07 +08:00
4d9f083e20
Add Switch component to UI and integrate it into DebugDrawer for tool state management. Update Assistants page to utilize the new Switch for enabling/disabling tools, enhancing user interaction and component functionality.
Xin Wang
2026-02-27 17:55:35 +08:00
531cf6080a
Update DuplexPipeline tool wait timeout to 60 seconds and modify DebugDrawer to improve tool call ID handling. Ensure better integration and functionality across components.
Xin Wang
2026-02-27 17:38:36 +08:00
229243e832
Add wait_for_response functionality to ToolResource and related components. Update API models, schemas, and routers to support new parameter. Enhance UI components to manage wait_for_response state, ensuring proper integration across the application.
Xin Wang
2026-02-27 16:54:39 +08:00
95c6e93a9c
Add text_msg_prompt tool to DuplexPipeline and Assistants. Update DebugDrawer to handle text message prompts, including parameter validation and state management for displaying messages. Ensure integration with existing tools and maintain functionality across components.
Xin Wang
2026-02-27 16:47:49 +08:00
cdd8275e35
Add voice_message_prompt tool to API and UI components. Update DuplexPipeline, Assistants, and DebugDrawer to support new tool functionality, including parameter validation and speech synthesis integration. Ensure existing tools are preserved during seeding process in the database.
Xin Wang
2026-02-27 16:04:49 +08:00
b035e023c4
Implement runtime tool ID and display name mapping in DuplexPipeline. Enhance Assistants and ToolLibrary components to utilize new mappings for improved tool identification and display. Update DebugDrawer to reflect changes in tool display names during interactions.
Xin Wang
2026-02-27 15:50:43 +08:00
0f1165af64
Enhance Dialog component to accept contentClassName prop for customizable styling. Update ToolLibraryPage to utilize new prop for improved modal layout. Refactor text in ToolLibrary for clarity.
Xin Wang
2026-02-27 15:12:01 +08:00
487634c494
Implement tool parameter management in ToolLibrary, including types, default value handling, and schema generation. Introduce functions for creating, updating, and removing tool parameters, enhancing the configuration capabilities for tools. Update state management to reflect new parameter structure.
Xin Wang
2026-02-27 15:04:52 +08:00
5f768edf68
Add parameter schema and defaults to ToolResource model and schemas. Implement runtime tool resolution in assistants and tools routers, ensuring proper handling of tool parameters. Update tests to validate new functionality and ensure correct integration of parameter handling in the API.
Xin Wang
2026-02-27 14:44:28 +08:00
d942c85eff
Add new tools to DuplexPipeline: calculator, code_interpreter, turn_on_camera, turn_off_camera, increase_volume, and decrease_volume. Implement fallback schema for unknown string tools and assign default client executors for specific tools. Update tests to validate new functionality and ensure correct tool handling in the pipeline.
Xin Wang
2026-02-27 13:59:37 +08:00
8d453e10da
Refactor AssistantsPage to utilize createPortal for template suggestion dropdowns, enhancing rendering performance and ensuring correct positioning relative to the document body. This change improves user experience by maintaining dropdown visibility during text input.
Xin Wang
2026-02-27 13:27:13 +08:00
e108ee66cb
Enhance AssistantsPage template suggestion functionality by adding control parameter to updateTemplateSuggestionState. This allows for dynamic positioning of suggestion dropdowns based on caret position, improving user experience during text input. Update relevant event handlers to pass control element for accurate suggestion placement.
Xin Wang
2026-02-27 12:18:48 +08:00
6178cc05bb
Add system-level dynamic variables support in session management. Implement methods to generate and apply built-in variables for current session time, UTC time, and timezone. Update documentation to reflect new variables and enhance tests for dynamic variable handling in the UI components.
Xin Wang
2026-02-27 12:08:18 +08:00
71cbfa2b48
Enhance DuplexPipeline and AssistantsPage for improved interruption handling. Introduce _OPENER_PRE_ROLL_MS constant for a head start on mic capture, and adjust interruption sensitivity settings from 500ms to 180ms across relevant components to optimize responsiveness during assistant interactions.
Xin Wang
2026-02-27 11:51:15 +08:00
6a9b5fcff4
Add error handling for dynamic variables in DebugDrawer component. Introduce state for dynamic variable errors and implement validation logic to manage required keys. Update methods to reset error state and handle errors during dynamic variable operations, enhancing user feedback and session management.
Xin Wang
2026-02-27 11:39:53 +08:00
3272a7a68a
Add dynamic variables support in session management and UI components. Implement validation rules for dynamic variables in metadata, including key format and value constraints. Enhance session start handling to manage dynamic variable errors. Update documentation and tests to reflect new functionality.
Xin Wang
2026-02-27 11:21:37 +08:00
f1b60bef22
Update ASR delta throttle timing in DuplexPipeline from 300ms to 500ms to improve processing efficiency and responsiveness.
Xin Wang
2026-02-27 10:23:06 +08:00
403b4b93c7
Add ASR capture timeout handling in DuplexPipeline and enhance EOU detection logic. Introduce _ASR_CAPTURE_MAX_MS constant and manage capture state timing to ensure timely end of utterance processing, even during silence. Update EouDetector to allow silence-only EOU when VAD state is lost.
Xin Wang
2026-02-27 09:59:54 +08:00
0b308f9bce
Remove deprecated agent configuration files: default.yaml, example.yaml, and tools.yaml, streamlining the agent behavior setup and eliminating unused parameters.
Xin Wang
2026-02-27 09:39:23 +08:00
e14eac347f
Update default.yaml configuration for speech agent parameters, adjusting min_speech_duration_ms from 100 to 120 ms and eou_threshold_ms from 800 to 1300 ms. Modify audio model parameters: set start_min_speech_ms to 100 ms, pre_speech_ms to 360 ms, and final_tail_ms to 180 ms for improved audio processing.
Xin Wang
2026-02-27 09:00:38 +08:00
0f02de5fc3
Update AssistantsPage to include new icon for audio preview button and adjust text for clarity. Add square icon for stop audio preview button to enhance UI consistency.
Xin Wang
2026-02-26 16:35:12 +08:00
0de6fe529e
Add audio preview functionality for assistant opener audio in AssistantsPage. Implement controls for previewing and stopping audio playback, and integrate new API endpoint for fetching PCM buffer. Enhance user interface with updated button states for audio actions.
Xin Wang
2026-02-26 16:15:31 +08:00
fb95e2abe2
Add opener audio functionality to Assistant model and related schemas, enabling audio generation and playback features. Update API routes and frontend components to support opener audio management, including status retrieval and generation controls.
Xin Wang
2026-02-26 14:31:50 +08:00
Xin Wang