StructuredOutputJudge

StructuredOutputJudge() compares the harness output with expected values from the judge config or matcher options. Use it for deterministic checks when the harness returns a parsed domain object instead of raw provider text.

Usage

evals/capital.eval.ts

import { StructuredOutputJudge } from "vitest-evals";

describeEval("capital questions", {
  harness: structuredQaHarness,
  judges: [
    StructuredOutputJudge({
      expected: { answer: "Paris" },
    }),
  ],
});

Explicit Assertion

Use explicit assertions when a case needs an extra check or when you want to record a score separately from the suite defaults.

evals/capital.eval.ts

it("records a structured output score", async ({ run }) => {
  const result = await run("What is the capital of France?");

  await expect(result).toSatisfyJudge(StructuredOutputJudge(), {
    expected: { answer: "Paris", country: "France" },
    threshold: null,
  });
});

threshold: null records the score without turning it into a failure.

Suite Expectations

Put suite-wide expected fields on the judge instance.

evals/capital.eval.ts

describeEval("capital questions", {
  harness: structuredQaHarness,
  judges: [
    StructuredOutputJudge({
      expected: {
        answer: "Paris",
        country: "France",
      },
    }),
  ],
});

Structured output checks work best when the harness returns a domain object instead of raw provider text.

Failure Behavior

The judge fails when the configured expected fields do not match the harness output and the score falls below the active threshold. Keep harness output as a domain object so reports can show field-level differences instead of raw text comparisons.