FactualityJudge

FactualityJudge(config?): Judge<FactualityJudgeOptions<any, any, any>>

Creates a factuality judge over normalized harness output.

FactualityJudge() compares input, output, and an expert answer. Bind a suite-wide expert answer on the judge config, or pass a case-specific expected value to toSatisfyJudge(...). Configure the LLM used for grading with judgeHarness on the judge, suite, or matcher options.

Parameters

config?

FactualityJudgeConfig = {}

Optional judge name and reusable judge harness default.

Returns

Judge<FactualityJudgeOptions<any, any, any>>

Example

import { anthropic } from "@ai-sdk/anthropic";
import { aiSdkJudgeHarness } from "@vitest-evals/harness-ai-sdk";
import { describeEval, FactualityJudge } from "vitest-evals";
import { qaHarness } from "./qaHarness";

const judgeHarness = aiSdkJudgeHarness({
  model: anthropic("claude-sonnet-4-5"),
  temperature: 0,
});
const factualityJudge = FactualityJudge({
  judgeHarness,
  expected: "Paris is the capital of France.",
});

describeEval("qa agent", {
  harness: qaHarness,
  judges: [factualityJudge],
}, (it) => {
  it("answers a geography question", async ({ run }) => {
    await run("What is the capital of France?");
  });
});