Third-party benchmark synthesis

Best AI for Code Review

A dated synthesis of public signals for AI code review, focusing on bug finding, tool use, and reviewer effort rather than generic chatbot polish.

Short answer

No single public leaderboard measures AI code review, so this page triangulates third-party signals: the Berkeley Function-Calling Leaderboard (updated 2026-04-12) for tool use, Aider's polyglot board (gpt-5 high at 88.0%) for multi-language code reasoning, and Terminal-Bench 2.1 (Claude Code + Claude 5 Fable at 83.1%) for harness discipline. Novamente reports these dated third-party results and does not publish its own code-review ranking.

Status: Public benchmark synthesis published; no first-party Novamente review run yet. This page reports public signals for code review workflows and blocks a Novamente ranking until seeded PR fixtures are reviewed.

Last updated: 2026-06-22. First-party tested: Not first-party tested.

Method: This page synthesizes public third-party benchmark signals and keeps Novamente out of first-party rankings until a dated run log exists. Figures in the copy below are attributed inline and dated.

Why no house ranking: rankings stay blocked until a first-party run log includes raw outputs or notes, failures, reviewer notes, and a retest date.

Download benchmark run log

Frozen benchmark fixtures
FixtureTaskExpected evidence
REVIEW-001 Review a PR with one obvious bug and one subtle edge-case bug. Finds both with line references.
REVIEW-002 Review a security-sensitive input path. Flags injection, validation, or escaping risks with evidence.
REVIEW-003 Review a clean PR with no seeded bug. Avoids noisy false positives.
35 Finding precision
30 Bug recall
20 Evidence quality
15 Reviewer effort

Code review has a benchmarking problem: there is no single public leaderboard that cleanly measures seeded-bug recall, false-positive control, and reviewer burden all at once. So this page has to be explicit about triangulation.

What the published evidence says (as of 2026-06)

On the Berkeley Function-Calling Leaderboard, last updated 2026-04-12, the benchmark defines tool use as a mix of function calling, multi-turn behavior, web search, memory, and format sensitivity. That matters for code review because the useful review assistant is often not a pure “chat reviewer” anymore. It needs to inspect files, keep context, and return structured findings that tooling can act on.

The BFCL page’s default comparison set, as observed in 2026-06, highlights Claude-Sonnet-4-20250514 (FC), GPT-5-2025-08-07 (FC), and Gemini-2.5-Pro (FC). I treat that as a signal that tool-use maturity is part of the review stack now, even when the benchmark is not code-review specific.

On Aider’s code editing leaderboard, the page itself now says it has been replaced by the newer polyglot board. That is useful context rather than a flaw: edit-format reliability still matters for review assistants that suggest fixes, but you should not mistake an older Python-edit benchmark for a complete review benchmark. It is a narrow signal about whether the model can apply changes cleanly.

On Aider’s polyglot leaderboard, as of 2026-06, gpt-5 (high) is listed at 88.0% correct across 225 exercises. I use that as a proxy for whether the model can reason about existing code in multiple languages, not as proof that it will catch review bugs with disciplined evidence.

On Terminal-Bench 2.1, as of 2026-06, Claude Code + Claude 5 Fable is at 83.1% while Terminus 2 + Claude 5 Fable is at 80.4%. Same model family, different harness, different outcome. For review tooling, that is the main lesson: wrapper behavior, file navigation, and command discipline can change reviewer usefulness as much as the base model.

How to read this page

If you need comment quality and structured findings, tool-use boards deserve more weight than general chatbot preference boards. If you need auto-fix suggestions, edit-format reliability is relevant, but only as one part of the picture. If you need low-noise review in a team setting, you still need seeded pull request fixtures because none of the public boards directly measure how many bad comments a reviewer has to clean up.

What our rubric still checks

Our rubric stays focused on the things public boards do not settle: did the assistant find the seeded bug, did it cite the right file and line, did it avoid noisy speculation, and did it reduce reviewer effort. Until we run that first-party fixture set, this page should be treated as a dated operator note, not a verdict.

If you need a local review process after reading the public signals, pair this page with AI Code Review Checklist and AI Code Review Workflow.