Open Source / MIT

PLUMBREF

Verify coding-agent answers
against your repo.

Setup once, ask in chat
installpipx install plumbref
setupplumbref init plumbref doctor --check-mcp-startup
chat"How does the nightly cleanup job work?" "Could this change affect downstream consumers?"
View on GitHubHow it worksRead docs
01 / Problem

THE GAP IN AGENT WORKFLOWS

Agents can explain code, but natural chat alone does not make the verification work visible. Plumbref lets you keep asking normally while the agent follows a bounded, source-grounded protocol behind the scenes.

01

Confident Answers, Thin Trail

You ask a useful repo question in chat, but the answer often does not preserve what was checked, what was found, or why a claim was accepted.

02

Confidence Without Citations

Agent explanations may sound right and mention files, but they usually do not anchor each important claim to specific source lines.

03

Repo Wandering

Letting an agent explore freely can use a lot of source context while still missing the contradiction pass or line-level evidence you actually needed.

Without Plumbref
  • -Agent answer with no durable source citations
  • -No record of what was searched
  • -No explicit contradiction pass
  • -Unbounded source reads unless the agent self-disciplines
  • -No structured report artifact
With Plumbref
  • +Same natural chat entrypoint
  • +Answer broken into checkable claims
  • +Claims linked to repository evidence
  • +Bounded search trace with file and line references
  • +Contradiction-pass tracking
  • +Budgets for searches, snippets, and reference depth
  • +Markdown and JSON reports
02 / How It Works

THREE STEPS

01

Initialize The Repo

Run one setup command in the repository. Plumbref writes a starter config, prints MCP JSON, and checks local readiness before you paste anything into an agent.

plumbref init plumbref doctor --check-mcp-startup
02

Ask Naturally

Ask questions like how a workflow works, what a field migration touches, or whether a change affects downstream consumers. The agent picks a template and turns the request into concrete claims.

"How does the nightly cleanup job work?" "What if we move customer_external_id?"
03

Verify Behind The Scenes

The agent uses MCP tools to search narrowly, read bounded snippets, run contradiction checks, record judgments, and return an answer-first report with uncertainty and limits.

plumbref_start → search_repo → read_evidence record_judgment → render_report
03 / Output

REPORT PREVIEW

Plumbref reports are readable first, auditable second: an answer up top, followed by claim checks, cited source snippets, search traces, and uncertainty.

report.md — verification output
MarkdownJSON
# Plumbref Report
Verdict: Partially supported
Verification mode: change_impact
Budget mode: normal
Template: Change impact (`change_impact` v1.0)
## Answer
Based on the checked evidence, the impact is: unchanged-but-unpushed records now enter the update path.
Important qualification: this is not an RTW-only guarantee; the condition is generic pushed=false.
## Verification Outcome
Answer gate: Answer with qualifications
Claim outcome: 4 supported, 1 qualified, 0 avoid
Safe to say: records with pushed=false are now re-enqueued when matched.
Say with qualification: external update success still depends on remote IDs and mappings.
## Measurement
Claims: 5 (supported=4, too_broad=1)
Searches: 23 run | Evidence snippets read: 13
Unsupported or qualified claims caught: 1
## Impact Claims
### supported
Matched existing records with pushed=false are included in changes.update.
services/remote_leaves.py:468-489 → pushed=false fallback
### too_broad
"Only affects RTW confirmations" needs narrower wording.
Safer wording: any matched record marked pushed=false can be re-enqueued.
## Search Trace
rg "changes.update" → 8 matches; rg "push_remote_leave_update" → 8 matches
Limits: source-backed for checked paths, not external API success.
status /supportedstatus /too_broadstatus /uncertainstatus /contradictedstatus /not_foundstatus /not_verifiable
04 / Features

WHAT IT DOES

Interface01

Agent-first MCP server

Run Plumbref as a stdio MCP tool server for Codex, Claude Code, and other MCP-capable agents.

Interface02

Natural chat workflow

Ask repo questions in chat. The agent uses Plumbref behind the scenes to verify claims against local source evidence.

Modes03

Reusable templates

Built-in playbooks guide common checks like flow explanation, field migration, change impact, downstream consumers, and external integrations.

Output04

Measured reports

Markdown and JSON reports include an answer, claim statuses, searches run, snippets read, contradiction passes, and qualified claims caught.

Search05

Local ripgrep search

All repository searches use ripgrep against local files. No embeddings, vector store, or external API calls.

Search06

Bounded source snippets

Evidence excerpts stay inside configurable file, line, and snippet budgets so the answer remains reviewable.

Budget07

fast / normal / deep modes

Control search depth per session, from quick checks to more thorough change-impact analysis.

Config08

Init and doctor

Run plumbref init to create starter config and print MCP setup, then plumbref doctor to check ripgrep, templates, reports, and MCP startup.

Semantics09

Conservative status semantics

Statuses default to the most conservative verdict supported by evidence, with contradiction-pass tracking.

05 / Comparison

WHY NOT JUST
ASK THE AGENT?

Coding agents can already reason over code.

Plumbref makes their evidence trail visible: claims, source snippets, and conservative verification reports that survive beyond the chat.

AspectAgent aloneWith Plumbref
Verification trailUsually not preserved beyond the conversation.Every claim, search, evidence snippet, and verdict is recorded.
Token scopeThe agent may search and read large chunks unless the prompt keeps it disciplined.Searches, snippets, and reference depth are bounded by explicit budgets.
Source citationsMay mention files, but important claims are not always anchored to lines.Evidence is bounded to specific file paths and line numbers.
Model involvementThe model reasons and judges in the same context.Plumbref does not call a model API. It records source evidence and conservative verdicts.
Report artifactNo report artifact is produced by default.Markdown and JSON reports include measurements, evidence, limits, and safer wording.
06 / Open Source

TRUST PROPERTIES

Property01

Local-first

Runs against files in your repository through your local agent setup. No Plumbref model API call is required.

Property02

Reviewable output

Produces Markdown and JSON reports that can be reviewed, shared, committed, or used in automation.

Property03

Auditable evidence

Every verdict is tied to the searched references, excerpts, contradiction checks, and limits used to judge a claim.

License

MIT

API keys required

None

Network calls

None

Hosted service

None

Database

None

Vector store

None

UI dependency

None

Configuration

TOML

Search engine

ripgrep

Python requirement

3.11+

Install method

pipx / pip

Setup helper

plumbref init

Health check

plumbref doctor

Data locality

Local repo only

Stated Limitations
  • 01

    Plumbref verifies repository evidence. It cannot confirm behavior that only exists in production data, runtime state, or external systems.

  • 02

    Broad questions are supported, but useful verification depends on concrete checks. Plumbref records the claims, searches, evidence, limits, and uncertainty behind each answer.

  • 03

    The agent still extracts claims and judges evidence. Plumbref records and constrains that work; it does not replace engineering judgment.

PLUMBREF

MIT License — contributions welcome

pipx install plumbref