The fastest way to lose a customer's trust isn't a wrong answer — it's a right answer in the wrong voice. A support reply that's technically correct but reads like a different company is a small betrayal, and at scale, small betrayals compound. Reply Judge exists to catch that betrayal before it ships.
Two models, two jobs
The responder drafts. The judge decides whether the draft sounds like you. We keep them separate on purpose: the model that's good at being helpful is not automatically good at being you, and conflating the two means you can't gate one without gating the other.
Reply Judge is anchored on your last ~200 sent, human-approved replies — the brand-voice embeddings. It scores a draft on fidelity to that anchor. Above the threshold, the reply goes out. Below it, the reply doesn't send; it lands in your inbox with the score and the specific lines that pulled it down.
{
"thread": "cs_thread#41207",
"draft_intent": "refund_partial",
"voice_fidelity": 0.71,
"threshold": 0.78,
"decision": "held_for_review",
"flagged_lines": ["We sincerely apologize for the inconvenience caused."],
"note": "anchor avoids canned-apology phrasing"
}That flagged line is grammatically perfect and completely off-brand for a store whose anchor replies are warm and plain-spoken. A single model optimizing for helpfulness would have sent it. The judge caught it at 0.71 and routed it to a human.
Why 0.78, and why you can move it
The default threshold is conservative because the cost of a bad-voice reply is asymmetric — one off reply costs more than ten held ones. But it's your dial. Tighten it during a launch when scrutiny is high; loosen it once the anchor has grown and the held rate is mostly false positives. Every held reply is also training: approve or edit it, and it becomes part of the anchor.
Automation that can't tell when it sounds wrong shouldn't be allowed to speak for you. The judge is permission to let it speak at all.
— Sam Okafor
The point isn't that the machine writes every reply. The point is that nothing goes out under your name that doesn't sound like your name. Reply Judge is the line between 'we automated support' and 'we let a stranger answer our customers.'