Forward Deployed Engineering Tutorial · AI Deployments · Lesson 21 of 24
Guardrails and Human Review
Keep AI deployments safe: input and output checks, permission-aware retrieval, confidence thresholds, audit trails and human approval.
Layers of protection
| Layer | Example |
|---|---|
| Input | Strip or flag personal data before sending it to a model; detect prompt injection in retrieved documents |
| Retrieval | Only retrieve documents the user is allowed to see |
| Output | Validate schema; block disallowed content; require citations |
| Decision | Confidence thresholds: auto-approve only high-confidence, low-risk cases |
| Human | Review queue for the rest; easy correction that feeds the eval set |
| Audit | Log inputs, retrieved sources, outputs and approvals (with redaction) |
cases = [("refund 20 EUR, duplicate charge", 0.97, 20), ("refund 4,800 EUR, chargeback", 0.95, 4800),
("refund 35 EUR, unclear reason", 0.61, 35)]
AUTO_CONFIDENCE, AUTO_MAX_AMOUNT = 0.9, 100
for text, confidence, amount in cases:
route = "auto-approve" if confidence >= AUTO_CONFIDENCE and amount <= AUTO_MAX_AMOUNT else "human review"
print(f"{text:34} conf {confidence:.2f} -> {route}")
Output
refund 20 EUR, duplicate charge conf 0.97 -> auto-approve refund 4,800 EUR, chargeback conf 0.95 -> human review refund 35 EUR, unclear reason conf 0.61 -> human review
Discuss these with the customer's risk and compliance teams early; they decide what may be automated.
Test yourself
An internal assistant must not show HR documents to engineers. Where is this enforced?
In the prompt, by asking the model not toIn retrieval, by filtering documents on the user's permissionsBy using a bigger modelIt can't be enforced
Show answer
Answer: In retrieval, by filtering documents on the user's permissions
Never rely on the model to keep secrets; don't retrieve what the user can't see.