Mason — the Provasign reference agent (incubating)
Mason exists to test a hypothesis the benchmarks made concrete: if completeness lives in the engine, a free local model can do real agentic coding. It is an experimental CLI and says so — everything it claims below is oracle-scored and published.
The problem: coding agents are built for frontier models. Point one at a local 30B and it forgets steering, re-types tool output lossily, re-derives solved traversals through grep, and claims success on work it never did. Prompts don’t fix any of that — the failure is in what the model is allowed to do, not what it’s told.
What Mason does: it makes the behaviors that keep agents accurate and cheap properties of the harness — structurally enforced, model-blind. The result, measured: a free local model completes change-impact and rename tasks at the same engine ceiling as a frontier model (benchmarks). The same principle now reaches localized bug fixes: Prism’s edit-ready source delivery hands the model verbatim, line-numbered code plus its callers and covering tests, and the harness — not the model — decides when to use it.
mason "Rename the Status method of the ResponseWriter interface to StatusCode.
Apply the plan including ambiguous edits, then verify with 'go build ./...'"
That task runs end-to-end — complete type-resolved rename plan, 24 edits applied, build verified — on a local model at $0.
Why a harness instead of steering
Measured across model tiers (research), steered agents fail in two ways prompts cannot fix: they relay engine results lossily (re-typing a site list drops sites), and they re-derive solved traversals through grep. Mason makes both structurally impossible:
| Harness property | What it enforces |
|---|---|
| Payload isolation | Graph-operation payloads (site lists, edit plans) render directly to you; the model receives only counts and flags. It cannot drop what it never holds. |
| Invocation wall | Tasks shaped like measured graph wins (renames, signature changes, “all callers”, dead code) are walled onto the graph tools for their first turn. |
| Harness-applied edits | Rename plans are applied by Mason with per-line drift checks — never re-typed by the model. |
| Honesty guard | If a task asked for a change and the working tree is untouched when the model claims success, the claim is rejected. |
| Secret redaction | Every tool result is scanned before it reaches the model; credentials become [REDACTED:kind], itemized by kind. |
| Plan mode | --plan makes the session read-only in the harness — mutating tools are refused, not discouraged. |
Grounding, refusal handling, deterministic compaction, permission policies,
per-task checkpointing with /undo, hooks, and LSP diagnostics at edit time
are harness properties too — the README
has the full list.
Any model
mason --model sonnet "task…" # friendly names: sonnet · haiku · opus · fable · gpt
mason --model ollama:qwen3-coder:30b "task…" # local via Ollama
mason --model lmstudio:qwen2.5-coder-32b "…" # LM Studio
mason --model oai:http://localhost:8000#m "…"# any OpenAI-compatible server
/model shows one numbered list — installed local models, downloadable
local models (filtered by your machine’s memory, one-keypress download,
including installing Ollama itself), a curated API shortlist, and the live
model list from any vendor you’ve keyed. Picking an API model without a
stored key starts a guided setup; keys land only in the OS keychain.
Local models are first-class, not a fallback. The measured result the design targets: task-shaped graph operations make completeness tier-invariant — recall 1.00 from a free qwen3-coder:30b up to a frontier model, across four languages (benchmarks).
The numbers so far — free local model, oracle-scored
| Scenario | Model ($0) | Result |
|---|---|---|
| Change-impact, jackson — 8 sites incl. callers not named after the target | qwen3-coder:30b | recall 1.00 (agent-scored, all 8 sites) |
| Change-impact, Guava — 310 sites | qwen3-coder:30b | 0.997 (the engine ceiling), 1 agent turn |
| Change-impact, Grafana (Go) — 93 sites | qwen3-coder:30b | 1.000, 1 turn |
| Change-impact, Django (Python) — 32 sites | qwen3-coder:30b | 1.000 recall, 1 turn |
End-to-end rename: type-resolved plan → 24 edits applied → go build verified |
local 14B | build green |
The control: the same local model driving general-purpose CLIs without task-shaped graph operations scored 0–1 out of 9 on the same task family (AB-LOCAL-CLIS). The model didn’t change — the harness and the tool altitude did.
Installation
# or via Go
go install github.com/provasign/mason/cmd/mason@latest
Then, in any repo:
mason # full-screen TUI: transcript, markdown, inline diffs, autocomplete
mason "task…" # one-shot
mason --json --yes "task…" # CI: one JSON object on stdout
Where it fits
Mason is the incubating reference agent for the two core projects. It drives multiple model providers, consumes Prism natively, and records an evidence trail through Shale when available. Mason is deliberately second-line until its end-to-end agent workflows earn the same independent evidence and adoption readiness as Prism and Shale.