Commit Graph

123 Commits

Author SHA1 Message Date
Xin Wang
531cf6080a Update DuplexPipeline tool wait timeout to 60 seconds and modify DebugDrawer to improve tool call ID handling. Ensure better integration and functionality across components. 2026-02-27 17:38:36 +08:00
Xin Wang
229243e832 Add wait_for_response functionality to ToolResource and related components. Update API models, schemas, and routers to support new parameter. Enhance UI components to manage wait_for_response state, ensuring proper integration across the application. 2026-02-27 16:54:39 +08:00
Xin Wang
95c6e93a9c Add text_msg_prompt tool to DuplexPipeline and Assistants. Update DebugDrawer to handle text message prompts, including parameter validation and state management for displaying messages. Ensure integration with existing tools and maintain functionality across components. 2026-02-27 16:47:49 +08:00
Xin Wang
cdd8275e35 Add voice_message_prompt tool to API and UI components. Update DuplexPipeline, Assistants, and DebugDrawer to support new tool functionality, including parameter validation and speech synthesis integration. Ensure existing tools are preserved during seeding process in the database. 2026-02-27 16:04:49 +08:00
Xin Wang
b035e023c4 Implement runtime tool ID and display name mapping in DuplexPipeline. Enhance Assistants and ToolLibrary components to utilize new mappings for improved tool identification and display. Update DebugDrawer to reflect changes in tool display names during interactions. 2026-02-27 15:50:43 +08:00
Xin Wang
0f1165af64 Enhance Dialog component to accept contentClassName prop for customizable styling. Update ToolLibraryPage to utilize new prop for improved modal layout. Refactor text in ToolLibrary for clarity. 2026-02-27 15:12:01 +08:00
Xin Wang
487634c494 Implement tool parameter management in ToolLibrary, including types, default value handling, and schema generation. Introduce functions for creating, updating, and removing tool parameters, enhancing the configuration capabilities for tools. Update state management to reflect new parameter structure. 2026-02-27 15:04:52 +08:00
Xin Wang
5f768edf68 Add parameter schema and defaults to ToolResource model and schemas. Implement runtime tool resolution in assistants and tools routers, ensuring proper handling of tool parameters. Update tests to validate new functionality and ensure correct integration of parameter handling in the API. 2026-02-27 14:44:28 +08:00
Xin Wang
d942c85eff Add new tools to DuplexPipeline: calculator, code_interpreter, turn_on_camera, turn_off_camera, increase_volume, and decrease_volume. Implement fallback schema for unknown string tools and assign default client executors for specific tools. Update tests to validate new functionality and ensure correct tool handling in the pipeline. 2026-02-27 13:59:37 +08:00
Xin Wang
8d453e10da Refactor AssistantsPage to utilize createPortal for template suggestion dropdowns, enhancing rendering performance and ensuring correct positioning relative to the document body. This change improves user experience by maintaining dropdown visibility during text input. 2026-02-27 13:27:13 +08:00
Xin Wang
e108ee66cb Enhance AssistantsPage template suggestion functionality by adding control parameter to updateTemplateSuggestionState. This allows for dynamic positioning of suggestion dropdowns based on caret position, improving user experience during text input. Update relevant event handlers to pass control element for accurate suggestion placement. 2026-02-27 12:18:48 +08:00
Xin Wang
6178cc05bb Add system-level dynamic variables support in session management. Implement methods to generate and apply built-in variables for current session time, UTC time, and timezone. Update documentation to reflect new variables and enhance tests for dynamic variable handling in the UI components. 2026-02-27 12:08:18 +08:00
Xin Wang
71cbfa2b48 Enhance DuplexPipeline and AssistantsPage for improved interruption handling. Introduce _OPENER_PRE_ROLL_MS constant for a head start on mic capture, and adjust interruption sensitivity settings from 500ms to 180ms across relevant components to optimize responsiveness during assistant interactions. 2026-02-27 11:51:15 +08:00
Xin Wang
6a9b5fcff4 Add error handling for dynamic variables in DebugDrawer component. Introduce state for dynamic variable errors and implement validation logic to manage required keys. Update methods to reset error state and handle errors during dynamic variable operations, enhancing user feedback and session management. 2026-02-27 11:39:53 +08:00
Xin Wang
3272a7a68a Add dynamic variables support in session management and UI components. Implement validation rules for dynamic variables in metadata, including key format and value constraints. Enhance session start handling to manage dynamic variable errors. Update documentation and tests to reflect new functionality. 2026-02-27 11:21:37 +08:00
Xin Wang
0f02de5fc3 Update AssistantsPage to include new icon for audio preview button and adjust text for clarity. Add square icon for stop audio preview button to enhance UI consistency. 2026-02-26 16:35:12 +08:00
Xin Wang
0de6fe529e Add audio preview functionality for assistant opener audio in AssistantsPage. Implement controls for previewing and stopping audio playback, and integrate new API endpoint for fetching PCM buffer. Enhance user interface with updated button states for audio actions. 2026-02-26 16:15:31 +08:00
Xin Wang
fb95e2abe2 Add opener audio functionality to Assistant model and related schemas, enabling audio generation and playback features. Update API routes and frontend components to support opener audio management, including status retrieval and generation controls. 2026-02-26 14:31:50 +08:00
Xin Wang
833cb0d4c4 Add unsaved changes confirmation dialog in AssistantsPage to enhance user experience when opening debug window 2026-02-26 14:17:01 +08:00
Xin Wang
fbbb2e0fee Add assistant snapshot management to track unsaved changes and enhance debug handling 2026-02-26 14:13:47 +08:00
Xin Wang
37b646186d Refactor DebugDrawer to store submitted session metadata and update resolved config view with merged metadata 2026-02-26 11:21:34 +08:00
Xin Wang
1bcf625f86 Enhance runtime metadata fetching by including assistant and app IDs, and defaulting channel to 'web_debug' 2026-02-26 11:04:54 +08:00
Xin Wang
f77f7c7531 Voice library support dashscope 2026-02-26 03:54:52 +08:00
Xin Wang
6744646390 Update frontend debug drawer 2026-02-26 02:23:23 +08:00
Xin Wang
72ed7d0512 Unify db api 2026-02-26 01:58:39 +08:00
Xin Wang
56f8aa2191 Fix talking voice error 2026-02-12 19:39:26 +08:00
Xin Wang
81ed89b84f Vendor can show more 2026-02-12 19:29:24 +08:00
Xin Wang
3c7efce80b Consistent library UI 2026-02-12 19:23:30 +08:00
Xin Wang
28ca003662 Use generated short id for llm asr tts 2026-02-12 19:05:50 +08:00
Xin Wang
ff3a03b1ad Use openai compatible as vendor 2026-02-12 18:44:55 +08:00
Xin Wang
0f9543d8a4 Voice add interface change 2026-02-12 18:01:05 +08:00
Xin Wang
39bcd67eac Update voice vad ui 2026-02-12 15:31:45 +08:00
Xin Wang
82521e7b90 Update assistants setting ui 2026-02-12 15:28:10 +08:00
Xin Wang
edcbc2cec7 Add first turn option 2026-02-12 15:23:32 +08:00
Xin Wang
56ca95c200 Improve UI 2026-02-12 15:09:25 +08:00
Xin Wang
a7ef8858de Fix frontend opener showing 2026-02-12 14:46:16 +08:00
Xin Wang
ef13ddb6b2 Text drawer use generated opener 2026-02-12 14:40:22 +08:00
Xin Wang
d41db6418c Add bot not interrupt and generated opener 2026-02-12 13:51:27 +08:00
Xin Wang
6179053388 Fix text and tool interleaving with minimal change 2026-02-11 15:32:43 +08:00
Xin Wang
6e63b49a4c Clean ovewview code 2026-02-11 15:01:59 +08:00
Xin Wang
44ad52669f Update overview layout 2026-02-11 14:55:43 +08:00
Xin Wang
2ab4075fcd Update overview ui 2026-02-11 14:46:06 +08:00
Xin Wang
24ec548924 Update overview ui 2026-02-11 14:28:41 +08:00
Xin Wang
ede12e2df0 Update overview 2026-02-11 14:14:56 +08:00
Xin Wang
e7605f661b Use pop window to add tool 2026-02-11 12:07:55 +08:00
Xin Wang
02d12ea996 Update assistant tool setting ui 2026-02-11 12:00:36 +08:00
Xin Wang
4c46793169 Make server tool http based 2026-02-11 11:39:45 +08:00
Xin Wang
9304927fe9 Import tool setting 2026-02-11 11:04:05 +08:00
Xin Wang
15523d9ec2 Add tts/text output schema 2026-02-11 09:50:46 +08:00
Xin Wang
6cac24918d Now we have server tool and client tool 2026-02-10 19:13:54 +08:00