Fix an issue where Grok Realtime would error out when running with SmallWebRTC transport.

The underlying issue was related to the fact that we were sending audio to Grok before we had configured the Grok session with our default input sample rate (16000), so Grok was interpreting those initial audio chunks as having its default sample rate (24000). We didn't see this issue when using the Daily transport simply because in our test environments Daily took a smidge longer than a reflexive (localhost) pure WebRTC connection, so we would only send audio to Grok *after* we had configured the Grok session with the desired sample rate.

changelog/3480.fixed.md

@@ -0,0 +1 @@
+- Fixed an issue where Grok Realtime would error out when running with SmallWebRTC transport.

src/pipecat/services/grok/realtime/llm.py

@@ -752,6 +752,14 @@ class GrokRealtimeLLMService(LLMService):
 
     async def _send_user_audio(self, frame):
         """Send user audio to Grok."""
+        # Don't send audio if conversation setup is still pending, as it can
+        # lead to errors. For example: audio sent before conversation setup
+        # will be interpreted as having Grok's default sample rate (24000),
+        # and if that differs from the sample rate we eventually set through
+        # the conversation setup, Grok will error out.
+        if self._llm_needs_conversation_setup:
+            return
+
         payload = base64.b64encode(frame.audio).decode("utf-8")
         await self.send_client_event(events.InputAudioBufferAppendEvent(audio=payload))