wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scripts(evals): update to use new runner function

Browse Source
This commit is contained in:
Aleix Conchillo Flaqué
2025-08-05 11:46:28 -07:00
parent 1138c92a00
commit 5546c8e01c
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
scripts/evals/README.md
View File

@@ -32,7 +32,7 @@ also explains why it thinks the answer is valid or invalid.
To run the release evals: To run the release evals:
```sh ```sh
python run-release-evals.py -a -v uv run run-release-evals.py -a -v
``` ```
This runs all the evals and stores logs and audio (`-a`) for each test. This runs all the evals and stores logs and audio (`-a`) for each test.
@@ -41,7 +41,7 @@ You can also specify which tests to run. For example, to run all `07` series
tests: tests:
```sh ```sh
python run-release-evals.py -p 07 -a -v uv run run-release-evals.py -p 07 -a -v
``` ```
## Script Evals ## Script Evals
@@ -49,7 +49,7 @@ python run-release-evals.py -p 07 -a -v
You can also run evals for a single example (not part of the release set): You can also run evals for a single example (not part of the release set):
```sh ```sh
python run-eval.py -p "A simple math addition" -a -v YOUR_EXAMPLE_SCRIPT uv run run-eval.py -p "A simple math addition" -a -v YOUR_EXAMPLE_SCRIPT
``` ```
Your script needs to follow any of the foundation examples pattern. Your script needs to follow any of the foundation examples pattern.

2
scripts/evals/eval.py
View File

@@ -176,7 +176,7 @@ async def run_example_pipeline(script_path: Path):
), ),
) )
await module.run_example(transport, argparse.Namespace(), True) await module.run_bot(transport)
async def run_eval_pipeline( async def run_eval_pipeline(