Customer Service
The conversation every turn so far, from both sides
The customer record who they are and what they have contacted us about before
The account subscription, recent charges, refunds and card status
A pending proposal whatever the assistant last offered and is waiting on
Output any that apply, sometimes none
SAY REFUND
FREEZE CARD
SET INTENT
HAND OFF
FLAG FOR REVIEW
CLOSE
Customer Service: accuracy against cost and time
| Type | Safe | OpenAI | Anthropic | Fireworks | workflow | prompt |
|---|---|---|---|---|---|---|
| frontier: nothing is both cheaper and more accurate | ||||||
| 10% | 30% | 50% | 70% | 90% | $0.0001 | $0.001 |
| cost per case, USD (log) | accuracy | |||||
| haiku 4.5 · workflow · 55.4% · $0.0074 · 8.8 | ||||||
| shaiku 4.5haiku 4.5 · prompt · 19.9% · $0.0459 · 7.6 | ||||||
| shaiku 4.5opus 5 · workflow · 72.4% · $0.0579 · 16.6 | ||||||
| sopus 5opus 5 · prompt · 66.0% · $0.3710 · 42.2 | ||||||
| sopus 5sonnet 5 · workflow · 69.3% · $0.0264 · 14.3 | ||||||
| ssonnet 5sonnet 5 · prompt · 61.6% · $0.2085 · 96.7 | ||||||
| ssonnet 5DS v4 flash · workflow · 76.8% · $0.0029 · 34.6 | ||||||
| sDS v4 flash · prompt · 65.8% · $0.0139 · 93.9 | ||||||
| sDS v4 flashDS v4 pro · workflow · 76.1% · $0.0232 · 58.6 | ||||||
| sDS v4 proDS v4 pro · prompt · 65.8% · $0.0898 · 134.9 | ||||||
| sDS v4 proluna · workflow · 71.4% · $0.0013 · 8.8 | ||||||
| sluna · prompt · 56.9% · $0.0094 · 19.2 | ||||||
| slunasol · workflow · 78.3% · $0.0323 · 10.1 | ||||||
| ssol · prompt · 69.0% · $0.2303 · 24.9 | ||||||
| ssolterra · workflow · 72.7% · $0.0111 · 6.0 | ||||||
| sterra · prompt · 64.4% · $0.0894 · 16.3 | ||||||
| sterraJev · workflow · 76.0% · $0.0001 · 0.4 | ||||||
| sDS v4 flashsolJev |
The workflow
1
Intent, frustration and risk flags
Eleven readings of the conversation at once: what the customer wants and how they want it resolved, how frustrated they sound, how urgent it is, and flags such as unauthorized activity, a legal threat, or a request for a person.
2
Consent, fraud, money, retention
Four follow-ups, plus a tie-break when two intents are close. Each runs only when the first reading calls for it, with the records it needs attached: the pending proposal, the ledger and card status, the refund or fee on file, the subscription.
3
Check the assistant's claims
The assistant's own messages are compared with the record: did it tell the customer a refund had gone through, or that the card was frozen, when neither had happened?
4
Safety first, then the issue at hand.
Nine sections run top to bottom: safety, the assistant's claims, a pending proposal, handing off, closing, then the issue itself. The first rule that applies in a section emits its actions; when none applies, the next section runs.
- Charged again after cancellation Opus differs from the other two
- Repeat duplicate charge root cause Sol differs from the other two
- Espresso machine impulse return refund TypeSafe differs from the other two
- Frustrated customer threatens to cancel All three miss the reference
- Overdraft fee hardship referral All three agree
Models
- Opus
- Sol
- TypeSafe
Closing
- resolved
- close
- closing acknowledgement
Recording the intent
- intent unclear
- ask clarify issue
- intent clear (the turn continues)
- set intent
- unauthorized activity: the investigation tree explained after all.