Refine prompts and evaluation cases for accident reporting AI
- Updated prompts to enhance clarity and consistency in user interactions, particularly focusing on safety prioritization and accurate information collection. - Introduced new YAML evaluation cases to test various user input scenarios, ensuring robust handling of short, ambiguous, or irrelevant responses. - Enhanced guidelines for transitioning to human assistance, emphasizing strict conditions for triggering state changes based on user responses.
This commit is contained in:
@@ -6,7 +6,7 @@ During the run it also prints the live dialogue and saves transcripts.
|
||||
|
||||
## Case format
|
||||
|
||||
Add YAML files anywhere under `evals/cases/`:
|
||||
Add YAML files anywhere under `evals/cases/` (e.g. `direct/`, `browser_addon/`, `no_false/` for false-interrupt / false-transfer checks):
|
||||
|
||||
```yaml
|
||||
name: direct_initial_stage
|
||||
@@ -45,6 +45,7 @@ Run selected cases:
|
||||
```bash
|
||||
python evals/run.py --case direct
|
||||
python evals/run.py --case browser_addon
|
||||
python evals/run.py --case no_false
|
||||
```
|
||||
|
||||
## Dialogue and transcripts
|
||||
|
||||
Reference in New Issue
Block a user