Session Helpers

Session helpers read a harness run. Use them in direct Vitest checks or inside custom judges through ctx.session and ctx.toolCalls.

Tool Calls

Use toolCalls(result) for the flattened tool history. It returns assistant tool-call requests with status, result, and error filled from matching type: "tool_result" transcript events that reference the call id. Calls without a matching result have status: "pending". The helpers also accept a normalized session when that is the only value available.

evals/refund.eval.ts

import { expect } from "vitest";
import { toolCalls } from "vitest-evals";

const calls = toolCalls(result);

expect(calls.map((call) => call.name)).toEqual([
  "lookupInvoice",
  "createRefund",
]);
expect(calls[0]).toMatchObject({
  name: "lookupInvoice",
  arguments: {
    invoiceId: "inv_123",
  },
});

Messages

Use role helpers for common transcript checks.

evals/refund.eval.ts

import {
  assistantMessages,
  latestAssistantMessageContent,
  messagesByRole,
  toolMessages,
  userMessages,
} from "vitest-evals";

expect(userMessages(result)[0]?.content).toBe("Refund invoice inv_123");
expect(latestAssistantMessageContent(result)).toContain("approved");
expect(assistantMessages(result).length).toBeGreaterThan(0);
expect(toolMessages(result)).toHaveLength(1);
expect(messagesByRole(result, "assistant")).toEqual(
  assistantMessages(result),
);

Read result.session.events directly when order across messages and tool activity matters.

evals/refund.eval.ts

expect(result.session.events.map((event) => event.type)).toEqual([
  "message",
  "tool_call",
  "tool_result",
  "message",
]);

API Links

Helper	Use
toolCalls	Flatten tool calls from a run.
assistantMessages	Read assistant transcript turns.
userMessages	Read user transcript turns.
systemMessages	Read system transcript turns.
toolMessages	Read tool transcript turns.
messagesByRole	Read messages for any normalized role.
latestAssistantMessageContent	Read the latest non-empty assistant content.