N Noer

Treat code-review-graph as an impact signal, not a correctness oracle

Noer code-review-graphAI code reviewMCP

A governance-oriented evaluation of local code graphs, benchmark limits, data boundaries, and PR review gates.

The expensive part of AI code review is often not the model rate. It is the amount of irrelevant repository context the model reads before it reaches the actual dependency edge. A small change can pull in a directory of similar files, tool output, and unrelated history, making the answer look informed while becoming harder to audit.

code-review-graph attacks that boundary. It parses a repository with Tree-sitter, stores structural relationships locally, and exposes blast-radius and review context through a CLI, MCP tools, and a GitHub Action.

A context gate before semantic search

The useful distinction is structural versus semantic retrieval. A graph can answer who calls a symbol, which modules depend on it, and where the tests are. Embeddings can retrieve related explanations. CRG is most useful when the structural pass narrows the search space before an agent reads prose or code.

Read the benchmark claims carefully

The current README reports roughly a 65x median per-question token reduction across six pinned open-source repositories, with a range of about 36x to 376x, and an average impact F1 around 0.693. These are useful engineering signals, not a guarantee that review quality is unchanged. Some ground truth is derived from the same graph traversal, so recall is circular in that mode. Teams should reproduce the benchmark and add their own missed-dependency cases.

Local-first still needs a data policy

The graph lives in .code-review-graph/, and the project is MIT-licensed with a no-telemetry position. Optional embeddings can still call external endpoints. Review the data boundary for identifiers, docstrings, summaries, CI logs, fork pull requests, permissions, and caches. “Source code stays on the runner” is narrower than “nothing leaves the organization.”

A safe pilot

pipx install code-review-graph
code-review-graph install --platform claude-code
code-review-graph build
code-review-graph status
code-review-graph detect-changes --brief
code-review-graph serve

Use a medium-sized repository with tests. Compare the human-expected impact set, the graph result, and the files the agent actually reads. Track missed callers, false positives, indexing latency, and token usage before enabling a risk-based merge gate.

Where it belongs

  • As a repository map and impact pre-filter for coding agents.
  • As a PR risk and test-gap signal, not an automatic approval system.
  • As a local-first option for teams that cannot use hosted code indexing.
  • Not as proof of runtime correctness: configuration, concurrency, data quality, and behavior still need tests and review.

The durable value is turning “what should the agent read first?” into a reviewable engineering step. CRG can reduce context waste, but its adoption should be judged by missed-dependency evidence and recovery controls, not by a single impressive token ratio.