frames: added VisionImageRawFrame

CHANGELOG.md

@@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- Added `VisionImageRawFrame`. This is an input image frame with an associated
+  text. It is usually processed by vision services (e.g. Moondream). The text
+  guides the vision service on how to analyze the image.
+
 - Added support for including images or audio to LLM context messages using
   `LLMContext.create_image_message()` and `LLMContext.create_audio_message()`.
   For example, when creating `LLMMessagesAppendFrame`:

src/pipecat/frames/frames.py

@@ -1305,6 +1305,24 @@ class UserImageRawFrame(InputImageRawFrame):
         return f"{self.name}(pts: {pts}, user: {self.user_id}, source: {self.transport_source}, size: {self.size}, format: {self.format}, request: {self.request})"
 
 
+@dataclass
+class VisionImageRawFrame(InputImageRawFrame):
+    """Raw image input frame to be analyzed by vision services.
+
+    This is just an image with an associated text describing how the vision
+    service should analyze the image.
+
+    Parameters:
+        text: Description of how the vision service should analyze the image.
+    """
+
+    text: str
+
+    def __str__(self):
+        pts = format_pts(self.pts)
+        return f"{self.name}(pts: {pts}, source: {self.transport_source}, size: {self.size}, format: {self.format}, text: {self.text})"
+
+
 @dataclass
 class InputDTMFFrame(DTMFFrame, SystemFrame):
     """DTMF keypress input frame from transport."""