Forward Deployed Engineering Tutorial · AI Deployments · Lesson 20 of 24

Evaluating AI Systems

Measure AI quality with a golden test set built from real customer cases, track it in CI, and decide release thresholds with the customer.

Create a free account to save this page, take notes and track your progress. Sign in

Why evals

Prompt edits, model upgrades and new documents change behaviour in ways you can't eyeball. An evaluation set of real cases with expected answers turns "it seems better" into numbers.

Building an eval set

50 to a few hundred real, anonymised examples, including hard and edge cases.
Expected outputs written or reviewed by customer experts.
Metrics that match the task: exact match or field accuracy for extraction, retrieval hit rate, human-graded or model-graded rubric scores for free text.
Run it on every change (prompt, model, retrieval) and compare with the last release.

Comparing two prompt versions on a golden setTry it yourself

golden = [{"id": 1, "expected": {"total": "1200.00", "currency": "EUR", "due": "2026-04-01"}},
          {"id": 2, "expected": {"total": "89.90", "currency": "USD", "due": "2026-03-15"}},
          {"id": 3, "expected": {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}}]
runs = {
    "prompt v1": [{"total": "1200.00", "currency": "EUR", "due": "2026-04-01"},
                  {"total": "89.90", "currency": "USD", "due": "2026-03-15"},
                  {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}],
    "prompt v2": [{"total": "1200.00", "currency": "EUR", "due": "2026-04-01"},
                  {"total": "89.9", "currency": "USD", "due": "2026-03-15"},
                  {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}],
}
for name, outputs in runs.items():
    correct = total = 0
    for case, out in zip(golden, outputs):
        for field, value in case["expected"].items():
            total += 1
            correct += out.get(field) == value
    print(f"{name}: field accuracy {correct / total:.1%}")
print("v2 'only reworded the instructions' but now drops trailing zeros: block the release and fix")

golden = [{"id": 1, "expected": {"total": "1200.00", "currency": "EUR", "due": "2026-04-01"}},
          {"id": 2, "expected": {"total": "89.90", "currency": "USD", "due": "2026-03-15"}},
          {"id": 3, "expected": {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}}]
runs = {
    "prompt v1": [{"total": "1200.00", "currency": "EUR", "due": "2026-04-01"},
                  {"total": "89.90", "currency": "USD", "due": "2026-03-15"},
                  {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}],
    "prompt v2": [{"total": "1200.00", "currency": "EUR", "due": "2026-04-01"},
                  {"total": "89.9", "currency": "USD", "due": "2026-03-15"},
                  {"total": "15000.00", "currency": "GBP", "due": "2026-05-30"}],
}
for name, outputs in runs.items():
    correct = total = 0
    for case, out in zip(golden, outputs):
        for field, value in case["expected"].items():
            total += 1
            correct += out.get(field) == value
    print(f"{name}: field accuracy {correct / total:.1%}")
print("v2 'only reworded the instructions' but now drops trailing zeros: block the release and fix")

Output

prompt v1: field accuracy 100.0%
prompt v2: field accuracy 88.9%
v2 'only reworded the instructions' but now drops trailing zeros: block the release and fix

Agree release thresholds with the customer ("field accuracy at least 95% and no regressions on critical fields") and keep monitoring in production with sampled human review.

Test yourself

What should an AI eval set be built from?

Real, anonymised customer cases with expert-reviewed expected outputs
Random internet text
Only easy examples
The model's own outputs

Show answer

Answer: Real, anonymised customer cases with expert-reviewed expected outputs

Representative real cases are what make eval results predictive.

Was this page helpful?