Workflow evaluations

The evaluation runner calls a live FastAPI service backed by FastGPT. For now, each conversation step checks only the SSE stage_code.nextStageCode value.

Case format

Add YAML files anywhere under evals/cases/:

name: browser_addon_example
clientMode: browser_addon

steps:
  - input: hi
    expect:
      stageCode: browser_addon.1001

  - input: 继续
    expect:
      stageCode: browser_addon.1002

Steps in one file share the same generated sessionId. Different cases always use different session IDs.

Run

Start the API, then execute:

python evals/run.py

Use another API address:

python evals/run.py --base-url http://192.168.1.10:8000

Run selected cases:

python evals/run.py --case browser_addon

The process exits with code 1 when an expectation fails and code 2 when case configuration is invalid.

903 B Raw Blame History

Workflow evaluations

Case format

Run

903 B

Raw Blame History