pipecat/THIRD_PARTY_INTEGRATIONS.md

Third-Party Integrations Guide

Pipecat welcomes community-maintained integrations! As our ecosystem grows, we've established a process for third-party developers to create and maintain their own service integrations while ensuring discoverability for the community.

Overview

What we support: Third-party maintained integrations that live in separate repositories and are maintained by their authors.

What we don't do: The Pipecat team does not review, test, or maintain third-party integrations. We provide guidance and list approved integrations for discoverability.

Why this approach: This allows the community to move quickly while keeping the Pipecat core team focused on maintaining the framework itself.

Requirements for Third-Party Integration Listing

To be listed as an official third-party integration, your repository must contain:

Required Components

• Source code - Complete implementation following Pipecat patterns
• Foundational example - Single file example showing basic usage (see Pipecat examples)
• README.md - Must include:
  • Introduction and explanation of your integration
  • Installation instructions
  • Usage instructions with Pipecat Pipeline
  • How to run your example
• LICENSE - Permissive license (BSD-2 like Pipecat, or equivalent open source terms)
• Code documentation - Source code with docstrings (we recommend following Pipecat's docstring conventions)
• Changelog - Maintain a changelog for version updates

Submission Information Required

When submitting your integration for listing, provide:

• Service name
• Service type (STT, LLM, TTS, Image, etc.)
• Link to your repository
• GitHub username(s) of maintainers

Submission Process

1. Create your integration following the patterns above
2. Set up your repository with all required components
3. Join our Discord: https://discord.gg/pipecat
4. Submit for listing in the #third-party-integrations channel with:
   • Service name and type
   • Repository link
   • Maintainer GitHub usernames

Integration Patterns and Examples

STT (Speech-to-Text) Services

Websocket-based Services

Base class: STTService

Examples:

• DeepgramSTTService
• SpeechmaticsSTTService

File-based Services

Base class: SegmentedSTTService

Examples:

• RivaSTTService
• FalSTTService

Key requirements:

• STT services should push InterimTranscriptionFrames and TranscriptionFrames
• If confidence values are available, filter for values >50% confidence

LLM (Large Language Model) Services

OpenAI-Compatible Services

Base class: OpenAILLMService

Examples:

• AzureLLMService
• GrokLLMService - Shows overriding the base class where needed

Non-OpenAI Compatible Services

Requires: Full implementation

Examples:

• AnthropicLLMService
• GoogleLLMService

TTS (Text-to-Speech) Services

AudioContextWordTTSService

Use for: Websocket-based services supporting word/timestamp alignment

Example:

• CartesiaTTSService

InterruptibleTTSService

Use for: Websocket-based services without word/timestamp alignment, requiring disconnection on interruption

Example:

• SarvamTTSService

WordTTSService

Use for: HTTP-based services supporting word/timestamp alignment

Example:

• ElevenLabsHttpTTSService

TTSService

Use for: HTTP-based services without word/timestamp alignment

Example:

• GoogleHttpTTSService

TTS Considerations:

• For websocket services, use asyncio WebSocket implementation (required for v13+ support)
• Handle idle service timeouts with keepalives
• TTSServices push both audio (TTSRawAudioFrame) and text (TTSTextFrame) frames

Telephony Services

Pipecat supports telephony provider integration using websocket connections to exchange MediaStreams. These services use a FrameSerializer to serialize and deserialize inputs from the FastAPIWebsocketTransport.

Examples:

• Twilio
• Telnyx

Considerations:

• Include hang-up functionality using the provider's native API, ideally using aiohttp

Image Generation Services

Base class: ImageGenService

Examples:

• FalImageGenService
• GoogleImageGenService

Requirements:

• Must implement run_image_gen method returning an AsyncGenerator

Implementation Guidelines

Naming Conventions

• STT: VendorSTTService
• LLM: VendorLLMService
• TTS:
  • Websocket: VendorTTSService
  • HTTP: VendorHttpTTSService
• Image: VendorImageGenService
• Vision: VendorVisionService
• Telephony: VendorFrameSerializer

Metrics Support

Enable metrics in your service:

def can_generate_metrics(self) -> bool:
    """Check if this service can generate processing metrics.

    Returns:
        True, as this service supports metrics.
    """
    return True

Dynamic Settings Updates

STT, LLM, and TTS services support ServiceUpdateSettingsFrame for dynamic configuration changes. The base STTService has an _update_settings() method that handles settings, and the private _settings Dict is used to store settings and provide access to the subclass.

async def set_language(self, language: Language):
    """Set the recognition language and reconnect.

    Args:
        language: The language to use for speech recognition.
    """
    logger.info(f"Switching STT language to: [{language}]")
    self._settings["language"] = language
    await self._disconnect()
    await self._connect()

Note that, in this example, Deepgram requires the websocket connection be disconnected and reconnected to reinitialize the service with the new value. Consider if your service requires reconnection.

Sample Rate Handling

Sample rates are set via PipelineParams and passed to each frame processor at initialization. The pattern is to not set the sample rate value in the constructor of a given service. Instead, use the start() method to initialize sample rates from the frame:

async def start(self, frame: StartFrame):
    """Start the service."""
    await super().start(frame)
    self._settings["output_format"]["sample_rate"] = self.sample_rate
    await self._connect()

Note that self.sample_rate is a @property set in the TTSService base class, which provides access to the private sample rate value obtained from the StartFrame.

Tracing Decorators

Use Pipecat's tracing decorators:

• STT: @traced_stt - decorate a function that handles transcript, is_final, language as args
• LLM: @traced_llm - decorate the _process_context() method
• TTS: @traced_tts - decorate the run_tts() method

Best Practices

Packaging and Distribution

• Use uv for packaging (encouraged)
• Consider releasing to PyPI for easier installation
• Follow semantic versioning principles
• Maintain a changelog

HTTP Communication

For REST-based communication, use aiohttp. Pipecat includes this as a required dependency, so using it prevents adding an additional dependency to your integration.

Error Handling

• Wrap API calls in appropriate try/catch blocks
• Handle rate limits and network failures gracefully
• Provide meaningful error messages

Testing

• Your foundational example serves as a valuable integration-level test
• Unit tests are nice to have. As the Pipecat teams provides better guidance, we will encourage unit testing more

Disclaimer

Third-party integrations are community-maintained and not officially supported by the Pipecat team. Users should evaluate these integrations independently. The Pipecat team reserves the right to remove listings that become unmaintained or problematic.

Questions?

Join our Discord community at https://discord.gg/pipecat and post in the #third-party-integrations channel for guidance and support.

For additional questions, you can also reach out to us at pipecat-ai@daily.co.