wx44wx/AI-VideoAssistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
194 Commits 2 Branches 0 Tags
229243e8321622efc250ef35f7b75f83102c5a42
Commit Graph

55 Commits

Author SHA1 Message Date
Xin Wang
229243e832 Add wait_for_response functionality to ToolResource and related components. Update API models, schemas, and routers to support new parameter. Enhance UI components to manage wait_for_response state, ensuring proper integration across the application. 2026-02-27 16:54:39 +08:00
Xin Wang
95c6e93a9c Add text_msg_prompt tool to DuplexPipeline and Assistants. Update DebugDrawer to handle text message prompts, including parameter validation and state management for displaying messages. Ensure integration with existing tools and maintain functionality across components. 2026-02-27 16:47:49 +08:00
Xin Wang
cdd8275e35 Add voice_message_prompt tool to API and UI components. Update DuplexPipeline, Assistants, and DebugDrawer to support new tool functionality, including parameter validation and speech synthesis integration. Ensure existing tools are preserved during seeding process in the database. 2026-02-27 16:04:49 +08:00
Xin Wang
b035e023c4 Implement runtime tool ID and display name mapping in DuplexPipeline. Enhance Assistants and ToolLibrary components to utilize new mappings for improved tool identification and display. Update DebugDrawer to reflect changes in tool display names during interactions. 2026-02-27 15:50:43 +08:00
Xin Wang
5f768edf68 Add parameter schema and defaults to ToolResource model and schemas. Implement runtime tool resolution in assistants and tools routers, ensuring proper handling of tool parameters. Update tests to validate new functionality and ensure correct integration of parameter handling in the API. 2026-02-27 14:44:28 +08:00
Xin Wang
d942c85eff Add new tools to DuplexPipeline: calculator, code_interpreter, turn_on_camera, turn_off_camera, increase_volume, and decrease_volume. Implement fallback schema for unknown string tools and assign default client executors for specific tools. Update tests to validate new functionality and ensure correct tool handling in the pipeline. 2026-02-27 13:59:37 +08:00
Xin Wang
71cbfa2b48 Enhance DuplexPipeline and AssistantsPage for improved interruption handling. Introduce _OPENER_PRE_ROLL_MS constant for a head start on mic capture, and adjust interruption sensitivity settings from 500ms to 180ms across relevant components to optimize responsiveness during assistant interactions. 2026-02-27 11:51:15 +08:00
Xin Wang
f1b60bef22 Update ASR delta throttle timing in DuplexPipeline from 300ms to 500ms to improve processing efficiency and responsiveness. 2026-02-27 10:23:06 +08:00
Xin Wang
403b4b93c7 Add ASR capture timeout handling in DuplexPipeline and enhance EOU detection logic. Introduce _ASR_CAPTURE_MAX_MS constant and manage capture state timing to ensure timely end of utterance processing, even during silence. Update EouDetector to allow silence-only EOU when VAD state is lost. 2026-02-27 09:59:54 +08:00
Xin Wang
fb95e2abe2 Add opener audio functionality to Assistant model and related schemas, enabling audio generation and playback features. Update API routes and frontend components to support opener audio management, including status retrieval and generation controls. 2026-02-26 14:31:50 +08:00
Xin Wang
da83c8ec8a Implement initial greeting emission in DuplexPipeline after session activation, ensuring proper event ordering for frontend notifications. 2026-02-26 14:07:46 +08:00
Xin Wang
562341a72c add dashscope tts 2026-02-26 03:02:48 +08:00
Xin Wang
72ed7d0512 Unify db api 2026-02-26 01:58:39 +08:00
Xin Wang
ff3a03b1ad Use openai compatible as vendor 2026-02-12 18:44:55 +08:00
Xin Wang
35bd83767e Cleanup engine 2026-02-12 17:42:21 +08:00
Xin Wang
aabf2ce8b9 Fix asr begin error 2026-02-12 16:52:42 +08:00
Xin Wang
543528239e Tune engine vad config 2026-02-12 16:29:55 +08:00
Xin Wang
a92a56b845 Presist opener to history 2026-02-12 15:59:36 +08:00
Xin Wang
399c9c97b1 Add tool call log 2026-02-12 15:44:01 +08:00
Xin Wang
edcbc2cec7 Add first turn option 2026-02-12 15:23:32 +08:00
Xin Wang
cbebfe1c7a Fix opener not trigger when tts disabled 2026-02-12 14:55:03 +08:00
Xin Wang
a7ef8858de Fix frontend opener showing 2026-02-12 14:46:16 +08:00
Xin Wang
ef13ddb6b2 Text drawer use generated opener 2026-02-12 14:40:22 +08:00
Xin Wang
d41db6418c Add bot not interrupt and generated opener 2026-02-12 13:51:27 +08:00
Xin Wang
80e1d24443 Organize tool scheme 2026-02-11 11:22:56 +08:00
Xin Wang
9304927fe9 Import tool setting 2026-02-11 11:04:05 +08:00
Xin Wang
180a69ca67 Fix delta final priority problem in backend 2026-02-11 10:25:46 +08:00
Xin Wang
15523d9ec2 Add tts/text output schema 2026-02-11 09:50:46 +08:00
Xin Wang
2d7fc2b700 Add server tool timeout protection 2026-02-10 19:17:45 +08:00
Xin Wang
6cac24918d Now we have server tool and client tool 2026-02-10 19:13:54 +08:00
Xin Wang
4b8da32787 Update code for tool call 2026-02-10 16:28:20 +08:00
Xin Wang
539cf2fda2 Move rag to llm service 2026-02-10 13:47:08 +08:00
Xin Wang
6b4391c423 Implement KB features with codex 2026-02-10 07:35:08 +08:00
Xin Wang
6a42c47700 Prettier EOU log 2026-02-09 19:56:47 +08:00
Xin Wang
3537a865f7 Merge branch 'master' of https://gitea.xiaowang.eu.org/wx44wx/AI-VideoAssistant 2026-02-09 19:18:24 +08:00
Xin Wang
b3e1c3d380 Use priority queue for vad event 2026-02-09 19:17:50 +08:00
Xin Wang
b34d500479 Merge branch 'master' of https://gitea.xiaowang.eu.org/wx44wx/AI-VideoAssistant 2026-02-09 18:29:56 +08:00
Xin Wang
0135f718f3 Fix asr boundary bug 2026-02-09 18:28:43 +08:00
Xin Wang
29d0b931eb better asr log 2026-02-09 18:22:00 +08:00
Xin Wang
b57d9f3a65 Fix race asr problem 2026-02-09 18:01:40 +08:00
Xin Wang
088943166c Update asr interim 2026-02-09 17:24:15 +08:00
Xin Wang
cbae28263b Merge branch 'master' of https://gitea.xiaowang.eu.org/wx44wx/AI-VideoAssistant 2026-02-09 16:30:52 +08:00
Xin Wang
29d8361ca9 Refactor duplicate stream tts adapter 2026-02-09 16:28:59 +08:00
Xin Wang
6648f1d478 Merge branch 'master' of https://gitea.xiaowang.eu.org/wx44wx/AI-VideoAssistant 2026-02-09 16:09:46 +08:00
Xin Wang
11016c04da Ignore short sentence TTS 2026-02-09 16:09:26 +08:00
Xin Wang
3b5d7d0d4d Merge branch 'master' of https://gitea.xiaowang.eu.org/wx44wx/AI-VideoAssistant 2026-02-09 16:05:06 +08:00
Xin Wang
45d7a41cff Log TTS sentence split 2026-02-09 16:04:15 +08:00
Xin Wang
ed044bd8ad Cleanup duplex pipeline 2026-02-09 16:00:32 +08:00
Xin Wang
fe05cf5d74 Remove fade-in on sentence beginning 2026-02-09 15:17:13 +08:00
Xin Wang
65aab79fb6 Improve delimiter 2026-02-09 15:11:35 +08:00