The problem
Letting an agent act automatically on every decision it makes, with no regard for how confident it actually was, means a genuinely uncertain call gets executed exactly as confidently as an obvious one. For anything with real consequences — refunds, deletions, sending a message on someone’s behalf — that’s a risk worth gating.
Expected outcome
By the end of this guide you will have a small gate that routes low-confidence decisions to a human instead of letting the agent act automatically, using whatever confidence score the model already returns.
Requirements and costs
- Node.js (version 18 or newer).
- Cost: free. No dependencies.
Step-by-step implementation
- Download the starter from the link below.
- Gate any decision that includes a confidence score:
const { gate } = require("./confidence-gate"); const result = gate({ action: "refund", confidence: 0.92 }, 0.8); // result.routed_to is "auto_execute" or "human_review" - Act only when
routed_to === "auto_execute"; otherwise queue it for a human to approve, withresult.reasonexplaining why. - Pick your threshold deliberately — higher for higher-stakes actions (a refund over A$500), lower for low-stakes, easily-reversible ones.
A real example
gate({ action: "refund", confidence: 0.92 }, 0.8)
// routed_to: "auto_execute"
gate({ action: "refund", confidence: 0.55 }, 0.8)
// routed_to: "human_review", reason: "confidence 0.55 below threshold 0.8"
gate({ action: "refund" }, 0.8) // no confidence field at all
// routed_to: "human_review", reason: "no confidence score provided"
Verification and troubleshooting
What we tested, on 5 October 2026 (record TNT-TEST-2026-10-05-025): a high-confidence decision (0.92 vs 0.8 threshold) correctly auto-executed; a low-confidence one (0.55) correctly routed to human review with the reason stated; and a decision missing a confidence field entirely correctly routed to human review too — confirming the gate fails safe by default rather than assuming high confidence when none was given.
Troubleshooting:
- Your model doesn’t return a confidence score at all. This gate will then always route to human review (by design) — you’d need to prompt the model explicitly to include one, or derive a proxy signal some other way, before this becomes useful for automatic execution.
- Everything routes to human review even at high confidence. Double check your threshold argument — it defaults to 0.8, which may be stricter than you intended.
- You want different thresholds for different action types. Call
gate()with a different threshold per action — it’s just a plain function argument, not a fixed global setting.
Limitations and alternatives
A model’s self-reported confidence score is not a guarantee of actual correctness — some models are poorly calibrated, confidently wrong at times. This gate reduces risk on the decisions the model itself flags as uncertain; it doesn’t catch a confidently-wrong decision the model never doubted.
Free resources
The download contains the gate module.
Download the confidence gate (ZIP, 1 KB)
Is there a paid version?
Not yet. This is the complete method. TNT Review has sold nothing to date and has no checkout, so any price we discuss in future should be read as a hypothesis, not an offer.