wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #1474 from pipecat-ai/khk/mem0-changelog

Browse Source 
Changelog entry for mem0 service
This commit is contained in:
Mark Backman
2025-03-29 18:02:32 -04:00
committed by GitHub
parent acddddc508 a978a5cd4a
commit 5d829fb6a9
4 changed files with 27 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
CHANGELOG.md
View File

@@ -9,13 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added ### Added
- Added `SmallWebRTCTransport`, a new P2P WebRTC transport. - Added `Mem0MemoryService`. Mem0 is a self-improving memory layer for LLM
- Created two examples in `p2p-webrtc`: applications. Learn more at: https://mem0.ai/.
- **video-transform**: Demonstrates sending and receiving audio/video with `SmallWebRTCTransport` using `TypeScript`.
Includes video frame processing with OpenCV.
- **voice-agent**: A minimal example of creating a voice agent with `SmallWebRTCTransport`.
- Added support to `ProtobufFrameSerializer` to send the messages from `TransportMessageFrame` and `TransportMessageUrgentFrame`. - Added `SmallWebRTCTransport`, a new P2P WebRTC transport.
- Created two examples in `p2p-webrtc`:
- **video-transform**: Demonstrates sending and receiving audio/video with
`SmallWebRTCTransport` using `TypeScript`. Includes video frame
processing with OpenCV.
- **voice-agent**: A minimal example of creating a voice agent with
`SmallWebRTCTransport`.
- Added support to `ProtobufFrameSerializer` to send the messages from
`TransportMessageFrame` and `TransportMessageUrgentFrame`.
- Added support for a new TTS service, `PiperTTSService`. - Added support for a new TTS service, `PiperTTSService`.
(see https://github.com/rhasspy/piper/) (see https://github.com/rhasspy/piper/)

1
README.md
View File

@@ -63,6 +63,7 @@ pip install "pipecat-ai[option,...]"
| Speech-to-Speech | [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) | `pip install "pipecat-ai[google]"` | | Speech-to-Speech | [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) | `pip install "pipecat-ai[google]"` |
| Transport | [Daily (WebRTC)](https://docs.pipecat.ai/server/services/transport/daily), [FastAPI Websocket](https://docs.pipecat.ai/server/services/transport/fastapi-websocket), [SmallWebRTCTransport](https://docs.pipecat.ai/server/services/transport/small-webrtc), [WebSocket Server](https://docs.pipecat.ai/server/services/transport/websocket-server), Local | `pip install "pipecat-ai[daily]"` | | Transport | [Daily (WebRTC)](https://docs.pipecat.ai/server/services/transport/daily), [FastAPI Websocket](https://docs.pipecat.ai/server/services/transport/fastapi-websocket), [SmallWebRTCTransport](https://docs.pipecat.ai/server/services/transport/small-webrtc), [WebSocket Server](https://docs.pipecat.ai/server/services/transport/websocket-server), Local | `pip install "pipecat-ai[daily]"` |
| Video | [Tavus](https://docs.pipecat.ai/server/services/video/tavus), [Simli](https://docs.pipecat.ai/server/services/video/simli) | `pip install "pipecat-ai[tavus,simli]"` | | Video | [Tavus](https://docs.pipecat.ai/server/services/video/tavus), [Simli](https://docs.pipecat.ai/server/services/video/simli) | `pip install "pipecat-ai[tavus,simli]"` |
| Memory | [mem0](https://docs.pipecat.ai/server/services/memory/mem0) | `pip install "pipecat-ai[mem0]"` |
| Vision & Image | [fal](https://docs.pipecat.ai/server/services/image-generation/fal), [Google Imagen](https://docs.pipecat.ai/server/services/image-generation/fal), [Moondream](https://docs.pipecat.ai/server/services/vision/moondream) | `pip install "pipecat-ai[moondream]"` | | Vision & Image | [fal](https://docs.pipecat.ai/server/services/image-generation/fal), [Google Imagen](https://docs.pipecat.ai/server/services/image-generation/fal), [Moondream](https://docs.pipecat.ai/server/services/vision/moondream) | `pip install "pipecat-ai[moondream]"` |
| Audio Processing | [Silero VAD](https://docs.pipecat.ai/server/utilities/audio/silero-vad-analyzer), [Krisp](https://docs.pipecat.ai/server/utilities/audio/krisp-filter), [Koala](https://docs.pipecat.ai/server/utilities/audio/koala-filter), [Noisereduce](https://docs.pipecat.ai/server/utilities/audio/noisereduce-filter) | `pip install "pipecat-ai[silero]"` | | Audio Processing | [Silero VAD](https://docs.pipecat.ai/server/utilities/audio/silero-vad-analyzer), [Krisp](https://docs.pipecat.ai/server/utilities/audio/krisp-filter), [Koala](https://docs.pipecat.ai/server/utilities/audio/koala-filter), [Noisereduce](https://docs.pipecat.ai/server/utilities/audio/noisereduce-filter) | `pip install "pipecat-ai[silero]"` |
| Analytics & Metrics | [Canonical AI](https://docs.pipecat.ai/server/services/analytics/canonical), [Sentry](https://docs.pipecat.ai/server/services/analytics/sentry) | `pip install "pipecat-ai[canonical]"` | | Analytics & Metrics | [Canonical AI](https://docs.pipecat.ai/server/services/analytics/canonical), [Sentry](https://docs.pipecat.ai/server/services/analytics/sentry) | `pip install "pipecat-ai[canonical]"` |

2
pyproject.toml
View File

@@ -64,6 +64,7 @@ langchain = [ "langchain~=0.3.20", "langchain-community~=0.3.20", "langchain-ope
livekit = [ "livekit~=0.22.0", "livekit-api~=0.8.2", "tenacity~=9.0.0" ] livekit = [ "livekit~=0.22.0", "livekit-api~=0.8.2", "tenacity~=9.0.0" ]
lmnt = [ "websockets~=13.1" ] lmnt = [ "websockets~=13.1" ]
local = [ "pyaudio~=0.2.14" ] local = [ "pyaudio~=0.2.14" ]
mem0 = [ "mem0ai~=0.1.76" ]
mlx-whisper = [ "mlx-whisper~=0.4.2" ] mlx-whisper = [ "mlx-whisper~=0.4.2" ]
moondream = [ "einops~=0.8.0", "timm~=1.0.13", "transformers~=4.48.0" ] moondream = [ "einops~=0.8.0", "timm~=1.0.13", "transformers~=4.48.0" ]
nim = [] nim = []
@@ -86,7 +87,6 @@ ultravox = [ "transformers~=4.48.0", "vllm~=0.7.3" ]
webrtc = [ "aiortc~=1.10.1", "opencv-python~=4.11.0.86" ] webrtc = [ "aiortc~=1.10.1", "opencv-python~=4.11.0.86" ]
websocket = [ "websockets~=13.1", "fastapi~=0.115.6" ] websocket = [ "websockets~=13.1", "fastapi~=0.115.6" ]
whisper = [ "faster-whisper~=1.1.1" ] whisper = [ "faster-whisper~=1.1.1" ]
mem0 = [ "mem0ai~=0.1.76" ]
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
# All the following settings are optional: # All the following settings are optional:

8
src/pipecat/services/whisper.py
View File

@@ -9,10 +9,10 @@
import asyncio import asyncio
from enum import Enum from enum import Enum
from typing import AsyncGenerator, Optional from typing import AsyncGenerator, Optional
from typing_extensions import TYPE_CHECKING, override
import numpy as np import numpy as np
from loguru import logger from loguru import logger
from typing_extensions import TYPE_CHECKING, override
from pipecat.frames.frames import ErrorFrame, Frame, TranscriptionFrame from pipecat.frames.frames import ErrorFrame, Frame, TranscriptionFrame
from pipecat.services.ai_services import SegmentedSTTService from pipecat.services.ai_services import SegmentedSTTService
@@ -332,6 +332,7 @@ class WhisperSTTService(SegmentedSTTService):
""" """
try: try:
from faster_whisper import WhisperModel from faster_whisper import WhisperModel
logger.debug("Loading Whisper model...") logger.debug("Loading Whisper model...")
self._model = WhisperModel( self._model = WhisperModel(
self.model_name, device=self._device, compute_type=self._compute_type self.model_name, device=self._device, compute_type=self._compute_type
@@ -456,10 +457,11 @@ class WhisperSTTServiceMLX(WhisperSTTService):
whisper_lang = self.language_to_service_language(self._settings["language"]) whisper_lang = self.language_to_service_language(self._settings["language"])
chunk = await asyncio.to_thread( chunk = await asyncio.to_thread(
mlx_whisper.transcribe, audio_float, mlx_whisper.transcribe,
audio_float,
path_or_hf_repo=self.model_name, path_or_hf_repo=self.model_name,
temperature=self._temperature, temperature=self._temperature,
language=whisper_lang language=whisper_lang,
) )
text: str = "" text: str = ""
for segment in chunk.get("segments", []): for segment in chunk.get("segments", []):