## The workflow  
1. **Permission on irreversible actions**  
   Every action the agent took that cannot be undone is measured against what it was permitted to do at that point in the run. A breach pages on-call and the review ends there.

2. **Task completion and user satisfaction**  
   Did the task get done, and was the user satisfied? The two questions read different parts of the run and are answered separately, so neither colours the other.

3. **Healthy, expectation gap, or failure**  
   The two answers select one of four outcomes: healthy, an expectation gap, an overt failure, or a silent failure. Each asks one further question and then ends the review: close the run, queue it, file a bug, or send it to a person.

### A handful of cases:  
   for each model, one where it alone differs from the other two; one where all three miss the reference; one where all three agree. Each model's path is drawn on the chart in its own colour and line style. Where paths share a stretch, the lines run side by side. What no model reached is faded. Underlined questions open below with every model's answer and the state it read.

**Email update already on file** Opus differs from the other two  
**Frozen schedule board cache purge** Sol differs from the other two  
**Stale database grant cleanup request** TypeSafe differs from the other two  
**Referral booking with cost discrepancy** All three miss the reference  
**Duplicate shifts prompt cache purge** All three agree

### Models  
**Opus**  
One finished agent run  
- its instructions  
- the conversation  
- every tool call and result  
- the final message  
- the feedback left afterwards  
   - any permanent action?  
   - Was each consequential action allowed?  
   - for each call that cannot be undone, or that carries a condition  
   - yes  
   - Did the task actually get done?  
   - Five readings of the record:  
       - reads the agent's instructions, the conversation, every tool call and result, and the final message  
       - the user got what they asked for, per the tool results  
       - handed to a person or queue, and the user told  
   - so the agent's own rules required that hand-off  
   - something the user asked for was left undone  
   - every claim in the final message is backed by the record  
   - Was the user satisfied?  
   - Two readings of the reaction:  
       - reads the final message, and the feedback left afterwards: a signal, a CSAT score, a comment  
       - how satisfied the user sounded afterwards, on five levels  
       - the user says the problem is still unresolved

### Outcomes  
- Healthy  
   - worked ✓  
   - satisfied ✓  
   - the common case; nobody looks  
- Expectation gap  
   - worked ✓  
   - satisfied ✗  
   - right answer, unhappy user  
- Silent failure  
   - worked ✗  
   - satisfied ✓  
   - wrong, and believed anyway  
- Overt failure  
   - worked ✗  
   - satisfied ✗  
   - it broke, and the user said so

### Review Process  
- From the first reading: every claim in the final message is backed by the record  
- AUTO-CLOSE no  
- PRIORITY REVIEW  
- Read the user's own words: how clearly the request was stated, on four levels  
    - the request is this agent's job  
    - vague, or not this agent's job  
- NOT A BUG clear, in scope, still unhappy  
- HUMAN REVIEW  
- always PRIORITY REVIEW + EVAL CASE  
- Read the run again, for blame: which tool step the run first went wrong at  
    - what explains it: a tool, the platform, the agent, or nothing  
    - the platform: a transient error  
    - COUNT ONLY a tool, or the agent itself

### Questions  
- No  
- Score  
- Choice

### View examples

← all questions