LLM-as-a-Verifier achieves state-of-the-art performance across coding, robotics, and medical domains: Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%).

LLM-as-a-Verifier as a dense reward signal improves the sample efficiency of both off-policy and on-policy RL algorithms. On LIBERO, LLM-as-a-Verifier achieves ≈ 1.8× higher sample efficiency than sparse reward baselines when fine-tuning a π₀ policy with SAC. On the MATH reasoning benchmark, it achieves ≈ 1.1× higher sample efficiency when fine-tuning Qwen3-8B with GRPO.

We observe a strong correlation between the chronological progression of code generation steps and the scores from LLM-as-a-Verifier. The successful trajectory of the pytorch-model-cli task follows a coherent sequence of events—Read model.py → Install g++ compiler → Install CPU-only torch → Update hidden_dim → DONE and exhibits consistently increasing verifier scores. In contrast, the failed trajectory is characterized by erroneous behaviors—it unnecessarily installs the large torchvision package, which exhausts the available disk space and hits a compilation error—resulting in significantly lower scores.

LLM-as-a-Verifier generates a smooth, temporally aligned progress signal over the course of robot rollout while remaining robust to failure modes such as incorrect intent or imprecise grasping. The example above illustrates the “Corn on Plate” task from the RoboRewardBench-OOD dataset.

The key finding is that most agents already “know” how to solve the tasks. On Terminal-Bench, if you repeatedly sample 100 trajectories per task, they can outperform Claude Mythos and nearly solve the entire benchmark. But the problem is that they don't know which one is correct, particularly when dealing with long-horizon tasks. While standard LLM-as-a-Judge can be used here, they fail to provide sufficiently fine-grained feedback. When comparing complex solutions, judges often assign the same score, resulting in a tie and failing to discriminate between them. Coarse scoring leads to 27% ties on Terminal-Bench V2.

Given a task prompt \(x\), a language model \(p_\theta\), a criterion \(c\), and two candidate trajectories \(\tau_i\) and \(\tau_j\), we construct scoring prompts and obtain their conditional distributions \(p_{\theta}(v \mid x,c,\tau_i)\) and \(p_{\theta}(v \mid x, c, \tau_j)\) by extracting the logprobs from <score_A> and <score_B> tags using the following prompt:

Rather than collapsing each distribution to a single discrete score, we approximate the reward of a trajectory as:

where \(C\) is the number of evaluation criteria, \(K\) is the number of repeated verifications, \(G\) is the number of score tokens (granularity level), \(p_{\theta}(v_g \mid x, c, \tau)\) is the probability assigned by model \(\theta\) to score token \(v_g\), and \(\phi(v_g)\) maps each score token to a scalar value.

To select the best of \(N\) candidates under a constrained verification budget, PPT compares each candidate against a small set of \(k \ll N\) pivots, cutting cost from \(\mathcal{O}(N^2)\) to \(\mathcal{O}(Nk)\).

We find that verification accuracy consistently improves as we scale across multiple dimensions: (1) the granularity of score tokens, (2) the number of repeated evaluations, and (3) the decomposition of evaluation criteria.

Left: The verifier achieves 74.7% at k=1 and improves to 77.5% at k=16, consistently outperforming the discrete judge across all evaluation budgets. A single-pass verifier already beats a heavily ensembled judge. Right: The judge produces ties in 26.7% of comparisons at k=1 due to coarse discrete scoring, while the continuous verifier yields zero ties.

We quantify progress tracking with the Value-Order Correlation (VOC) — the Spearman rank correlation between a step's chronological index and the verifier's predicted value for the prefix ending at that step. A verifier that tracks progress assigns monotonically higher scores to later prefixes of a successful rollout (\(\mathrm{VOC}\to 1\)):

The fine-grained verifier score is a drop-in dense reward for both off-policy and on-policy RL, improving sample efficiency by ≈1.8× on LIBERO and ≈1.1× on MATH.

On RoboRewardBench, the verifier must integrate visual context across multi-frame videos to reason about physical progress. Applied zero-shot with a Qwen 3.6 35B VLM, LLM-as-a-Verifier reaches 87.4% trajectory preference accuracy—outperforming the discrete judge (70.8%) and reward models trained specifically on robotics data, including RoboReward-8B (81.4%), Robometer-4B (78.8%), and TOPReward (74.7%)—despite using no fine-tuning. It also reduces MAE against human annotations from 1.11 to 0.72.

Some frontier models (e.g., GPT-5.5, Claude Opus) expose only sampled completions and withhold token-level logprobs, which our continuous reward requires. A simple two-stage workaround recovers most of the calibrated signal: the closed model supplies domain-specific reasoning, and an open verifier supplies the calibrated probability distribution it withholds.

On Terminal-Bench V2, routing GPT-5.5's reasoning through Gemini 2.5 Flash recovers a +5.2-point accuracy gain over directly using the closed model's integer scores (80.1% vs. 74.9%) and eliminates its 10.9% tie rate entirely — without any access to the frontier model's logits.