Anthropic tool use core Pipecat pieces refactored (#369)

* processors(rtvi): rtvi 0.1 message protocol

* added a single function call handler

* wip - function calling

* fixup

* fixup

* fixup

* processors(rtvi): no need for configure_on_start()

* processors(rtvi): add new option values if they haven't been set yet

* Add the model name to the LLM usage metrics

* wip - anthropic tool calling

* still wip - anthropic tool use and vision

* anthropic tools and vision working

* anthropic tool calling and vision

* Cartesia error handling

* Anthropic tool use core Pipecat pieces refactored as per plan

* aleix has good ideas

* Usage metrics for Anthropic LLMs

* fix function call result state not getting cleared bug

* Pass **kwargs through from AnthropicLLMService constructor

* about to tinker with anthropic

* added openai function calling

* openai function calling

* fixup

---------

Co-authored-by: Aleix Conchillo Flaqué <aleix@daily.co>
Co-authored-by: Chad Bailey <chadbailey@gmail.com>
Co-authored-by: mattie ruth backman <mattieruth@gmail.com>
Co-authored-by: chadbailey59 <chadbailey59@users.noreply.github.com>

src/pipecat/frames/frames.py

@@ -4,7 +4,7 @@
 # SPDX-License-Identifier: BSD 2-Clause License
 #
 
-from typing import Any, List, Mapping, Tuple
+from typing import Any, List, Mapping, Tuple, Optional
 
 from dataclasses import dataclass, field
 
@@ -177,6 +177,15 @@ class LLMMessagesUpdateFrame(DataFrame):
     messages: List[dict]
 
 
+@dataclass
+class LLMSetToolsFrame(DataFrame):
+    """A frame containing a list of tools for an LLM to use for function calling.
+    The specific format depends on the LLM being used, but it should typically
+    contain JSON Schema objects.
+    """
+    tools: List[dict]
+
+
 @dataclass
 class TTSSpeakFrame(DataFrame):
     """A frame that contains a text that should be spoken by the TTS in the
@@ -389,6 +398,7 @@ class TTSStoppedFrame(ControlFrame):
 class UserImageRequestFrame(ControlFrame):
     """A frame user to request an image from the given user."""
     user_id: str
+    context: Optional[any]
 
     def __str__(self):
         return f"{self.name}, user: {self.user_id}"
@@ -406,3 +416,22 @@ class TTSVoiceUpdateFrame(ControlFrame):
     """A control frame containing a request to update to a new TTS voice.
     """
     voice: str
+
+
+@dataclass
+class FunctionCallInProgressFrame(SystemFrame):
+    """A frame signaling that a function call is in progress.
+    """
+    function_name: str
+    tool_call_id: str
+    arguments: str
+
+
+@dataclass
+class FunctionCallResultFrame(DataFrame):
+    """A frame containing the result of an LLM function (tool) call.
+    """
+    function_name: str
+    tool_call_id: str
+    arguments: str
+    result: any