wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #3360 from pipecat-ai/mb/openai-realtime-send-image

Browse Source 
Add video input (e.g. image input) support for OpenAI Realtime
This commit is contained in:
Mark Backman
2026-01-08 13:26:35 -05:00
committed by GitHub
parent 1098394486 2aadac7a4d
commit 5e0e6822c7
17 changed files with 257 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
changelog/3360.added.md Normal file
View File

@@ -0,0 +1,8 @@
- Added image support to `OpenAIRealtimeLLMService` via `InputImageRawFrame`:
- New `start_video_paused` parameter to control initial video input state
- New `video_frame_detail` parameter to set image processing quality ("auto",
"low", or "high"). This corresponds to OpenAI Realtime's `image_detail`
parameter.
- `set_video_input_paused()` method to pause/resume video input at runtime
- `set_video_frame_detail()` method to adjust video frame quality dynamically
- Automatic rate limiting (1 frame per second) to prevent API overload