Mike Seese
dc8ea615d9
add hathora to run-release-evals.py
2026-01-17 10:33:58 -08:00
Mark Backman
efd4432cfb
Renumber the 07 foundational examples
2026-01-15 10:26:17 -05:00
Aleix Conchillo Flaqué
248dac3a9d
Merge pull request #3420 from pipecat-ai/pk/fix-gemini-3-parallel-function-calls
...
Fix parallel function calling with Gemini 3.
2026-01-13 14:40:33 -08:00
Mark Backman
41eef5efc4
Add 07j Gladia VAD foundational example, add to release evals
2026-01-13 11:36:15 -05:00
Paul Kompfner
6668712f7b
Add evals for parallel function calling
2026-01-13 11:03:38 -05:00
Aleix Conchillo Flaqué
5da1f86575
scripts: add 53-concurrent-llm-evaluation.py to release evals
2026-01-09 09:26:38 -08:00
Mark Backman
4d61c5d7b2
Deprecate support for vad_events in DeepgramSTTService
2026-01-08 20:32:30 -05:00
Mark Backman
3a7b489208
Add foundational 19c and add to evals
2026-01-08 13:00:45 -05:00
Mark Backman
98f70b775f
Update copyright date range to 2024-2026
2026-01-07 16:58:13 -05:00
Mark Backman
31907b90f0
Add 07 example variants to release evals
2025-12-31 09:11:00 -05:00
Mark Backman
845b4ad20e
Add 51 foundational to evals
2025-12-20 08:07:25 -05:00
Mark Backman
56c58f7302
Move Ultravox foundational example to 50, add to release evals
2025-12-18 13:38:12 -05:00
Aleix Conchillo Flaqué
d07b37b288
scripts(evals): more eval prompts improvements
2025-12-17 09:55:12 -08:00
Aleix Conchillo Flaqué
5b30f1b1ef
scripts(evals): improve prompts
2025-12-16 17:26:50 -08:00
Aleix Conchillo Flaqué
5f7dbfe775
scripts(evals): don't use on_idle_timeout
2025-12-16 17:26:42 -08:00
Mark Backman
bd3bf9a00e
Inworld TTS services: Add websocket TTS class, add word-timestamp alignment
2025-12-16 13:47:24 -05:00
Aleix Conchillo Flaqué
21e346abe2
scripts(evals): improve eval prompts
2025-12-15 13:21:40 -08:00
Aleix Conchillo Flaqué
70a80847a7
scripts(evals): use future instead of a queue to store eval result
2025-12-15 13:21:28 -08:00
Aleix Conchillo Flaqué
a14c911fb2
scripts(evals): improve eval assertion on exit
2025-12-14 12:37:05 -08:00
Aleix Conchillo Flaqué
4f848e9631
Merge pull request #3227 from fixie-ai/mike/upstream
...
Add Ultravox service
2025-12-13 18:29:02 -08:00
Mike Depinet
4b81be7acf
Add Ultravox service ( #1 )
...
Adds support for using Ultravox Realtime as a speech-to-speech service.
Also removes the deprecated Ultravox speech-to-text vllm model integration to avoid confusion.
2025-12-12 10:16:15 -08:00
Paul Kompfner
12979293ad
Add thinking examples to eval suite
2025-12-11 15:58:48 -05:00
vipyne
acba544e6f
pr notes for nvidia service name change
2025-12-01 22:41:17 -06:00
Aleix Conchillo Flaqué
51ba245e10
scripts(evals): fix EVAL_CONVERSATION/EVAL_WEATHER eval
2025-11-18 21:14:27 -08:00
Aleix Conchillo Flaqué
38aac44a1e
scripts(evals): 26c should be a camera eval
2025-11-07 11:30:41 -08:00
Aleix Conchillo Flaqué
4f1468e0fa
scripts(evals): improve eval prompt
2025-11-07 10:05:46 -08:00
Mark Backman
7eb880c5e8
Add DeepgramHttpTTSService
2025-10-31 11:39:32 -04:00
Aleix Conchillo Flaqué
19f046a338
examples(foundational): add 12d-describe-image-moondream
2025-10-30 14:02:17 -07:00
Aleix Conchillo Flaqué
74fb6e7676
scripts(evals): improve eval prompting
2025-10-30 13:08:15 -07:00
Aleix Conchillo Flaqué
a997655eac
scripts(evals): simplify eval configuration and allow RunnerArgs body
2025-10-30 13:08:15 -07:00
Aleix Conchillo Flaqué
e458d3edfe
scripts(evals): update 12-* for 14-*-video
2025-10-30 13:08:15 -07:00
Mark Backman
8902f1dc94
Bump openai, openpipe versions, add 14x foundational example
2025-10-15 15:17:22 -04:00
Mark Backman
4b415721e2
Update release evals for OpenAI Realtime, Gemini Live Vertex; shorten 26 foundational names
2025-10-10 12:26:23 -04:00
Mark Backman
eddce460da
Deprecate PlayHT TTS services
2025-10-07 16:40:01 -04:00
ivaaan
1b511557b2
upd evals
2025-10-02 13:48:30 -07:00
ivaaan
4ffdabcfde
add Hume example, small fixes
2025-10-02 13:38:36 -07:00
Filipi Fuchter
11bc4ea854
Adding deepgram flux to release evals.
2025-10-01 19:24:58 -03:00
Paul Kompfner
6faa50ae5b
Remove remaining usage of OpenAILLMContext throughout the codebase in favor of LLMContext, except for:
...
- Usage in classes that are already deprecated
- Usage related to realtime LLMs, which don't yet support `LLMContext`
- Usage in (soon-to-be-deprecated) code paths related to `OpenAILLMContext` itself and associated machinery
2025-09-24 16:35:03 -04:00
Aleix Conchillo Flaqué
620b1f785c
examples: update Strands Agents with universal context and add evals
2025-09-23 11:37:57 -07:00
Paul Kompfner
272532a3ea
Update examples, wherever possible, to use LLMContext and associated machinery instead of OpenAILLMContext and associated machinery.
...
With all these examples updated, we no longer need dedicated examples illustrating `LLMContext`, so they're removed.
Here’s where we *don’t* yet use `LLMContext` and associated machinery:
- Realtime services: OpenAI Realtime, Gemini Live, and AWS Nova Sonic (support coming soon)
- `GoogleLLMOpenAIBetaService` (it’s deprecated, so we didn’t bother adding support)
- `LLMLogObserver` (support coming soon)
- `GatedOpenAILLMContextAggregator` (support coming soon)
- `LangchainProcessor` (support coming soon)
- `Mem0MemoryService` (support coming soon)
- Examples that use LLM-specific tools definitions as opposed to `ToolsSchema` (these will be updated soon)
- Examples that rely `GoogleLLMContext.upgrade_to_google` (TBD what to do with these)
Examples that use `LLMLogObserver`:
- 30-
Examples that use `GatedOpenAILLMContextAggregator`:
- 22-
Examples that use `LangchainProcessor`:
- 07b-
Examples that use `Mem0MemoryService`:
- 37-
Examples that need updating to use `ToolsSchema`:
- 15-
- 15a-
- 20a-
- 20c-
- 20d-
- 22b-
- 22c-
- 33-
- 36-
Examples that use `GoogleLLMContext.upgrade_to_google`:
- 22d-
- 25-
2025-09-22 16:21:35 -04:00
Aleix Conchillo Flaqué
82478be861
scripts(evals): add 19b-openai-realtime-text
2025-09-12 11:03:32 -07:00
Paul Kompfner
a4023d3915
Update evals to include examples that exercise the universal LLMContext
2025-09-11 14:32:56 -04:00
Aleix Conchillo Flaqué
46deaada70
scripts(evals): allow user to talk and only eval when needed
2025-09-06 19:19:08 -07:00
Mark Backman
977a57c8fb
Add 14k (CerebrasLLMService) to release evals
2025-09-03 17:11:38 -04:00
Aleix Conchillo Flaqué
aeb9f1ffca
transports: reorganize module
2025-09-02 17:31:39 -07:00
Aleix Conchillo Flaqué
5ebcea2a3b
scripts(eval): change "result" function call parameter
2025-08-28 11:38:59 -07:00
Paul Kompfner
189749b579
Add LLMRunFrame to trigger an LLM response, replacing context_aggregator.user().get_context_frame()
2025-08-28 09:53:33 -04:00
Aleix Conchillo Flaqué
a0128516ff
PipelineTask: handle cancellations gracefully
2025-08-22 19:04:31 -07:00
Aleix Conchillo Flaqué
9273ec0f25
scripts(evals): always require an eval prompt
2025-08-22 16:57:47 -07:00
Mark Backman
b30af3e155
Tests specify USER_SPEAKS_FIRST or BOT_SPEAKS_FIRST
2025-08-22 12:12:17 -04:00