Grading at Volume with LLM-as-Judge
Automating evaluation with model graders, prioritizing many imperfect evals over few perfect ones, and calibrating judges against human spot checks.
The grading ladder: automate whenever possible
Once you have success criteria and test cases, you need to grade outputs — hundreds of them, after every prompt change. Grading by hand doesn't scale, so the rule is: automate whenever possible, and treat human grading as the slow last resort.
Pick the cheapest grading method that fits the output type:
| Output type | Grading method |
|---|---|
| Categorical answer ("refund" / "billing" / "technical") | Exact match or string match — trivial code, zero ambiguity |
| Verifiable output (code, SQL, math) | Code-graded — run it and check the result |
| Consistency across runs | Cosine similarity — a number measuring how close two texts are in meaning |
| Summaries vs references | ROUGE-L — overlap with a reference summary |
| Anything fuzzy (tone, helpfulness, coherence) | LLM-as-judge — a model grades the output against a rubric |
| Truly novel judgment calls | Human grading — accurate but slow; use sparingly |
LLM-as-judge means using a language model as the grader: you show it the output plus a rubric, and it returns a score. It's the workhorse for everything that can't be string-matched, because it's fast, cheap, and consistent enough at scale.
Volume beats polish
A counterintuitive principle from Anthropic's guidance: more questions with slightly lower-signal automated grading beats a few hand-graded gems. Twenty lovingly hand-scored examples tell you less than 200 automatically judged ones, because the noise in any single grade averages out over volume, while a tiny set can't represent your real input distribution at all. Don't wait for the perfect eval — build a big imperfect one and improve it as you go.
Writing a good judge prompt
Four practices make model graders reliable:
- Give a detailed rubric, not a vague ask. "Rate coherence 1–5 based on argument flow and logical consistency" works; "is this good?" doesn't.
- Encourage reasoning first, then strip it. Have the judge think through its assessment before scoring — the reasoning improves the score's quality — then discard the reasoning and keep only the number.
- Demand machine-readable output: "After your reasoning, output only the number on the final line." Your harness needs to parse thousands of these.
- Use a different model as grader than generator. A model judging its own output has the same blind spots that produced the output.
A complete judge prompt looks like this:
You are grading a customer support summary.
<rubric>
Rate coherence 1-5:
5 = clear narrative, no contradictions, easy to follow
3 = understandable but disorganized or repetitive
1 = confusing, contradictory, or missing the point
</rubric>
<summary_to_grade>
{summary}
</summary_to_grade>
Think through your assessment against the rubric first.
Then output only the number 1-5 on the final line.
Binary vs scale judgments
Match the score shape to the question. Binary yes/no judges work best for safety and privacy criteria — "does this output contain personal information, yes or no?" has a right answer and needs no gradations. Likert scales (ordinal ratings like 1–5) suit quality gradations such as coherence or tone, where "how good" is the real question. Don't use a 1–5 scale for a yes/no question — it just adds noise.
Calibrate the judge against humans
An automated judge is itself a model output, so it needs its own verification. The method is spot checking: periodically grade a sample of outputs by hand and compare your grades to the judge's. If you and the judge disagree often, fix the rubric — usually it's missing a distinction you care about — and re-check. A judge you've calibrated once and never again will drift as your prompts and inputs evolve.
This is calibration, not full human grading: you hand-grade 20 samples occasionally to keep the automated grading of 2,000 honest.
A production example
Anthropic's multi-agent research system graded outputs with a single LLM-judge call scoring five dimensions — factual accuracy, citation accuracy, completeness, source quality, and tool efficiency — each 0.0–1.0, plus an overall pass/fail. Tested against alternatives, the single-call judge was "the most consistent and aligned with human judgements," and it scaled to grading hundreds of outputs per experiment. You don't need a complex grading pipeline; you need one well-rubric'd call.
Cover the ugly cases
Whatever the grading method, the eval set itself must include standard cases, edge cases, and adversarial cases — normal inputs, weird-but-legal inputs (empty, enormous, ambiguous), and inputs actively trying to break your rules. A judge can only grade what you show it; if the eval set is all sunny-day inputs, a perfect score means nothing about production.