NAI OS
Conventions, and the price of hardening.
Two documents carry the conventions a new system is built to. CLAUDE.md points at
both: “Agent prompt style: docs/agent-conventions.md”, and “How the
systems verify their own output — four shapes, from a bounded loop to no check at all —
is mapped in docs/verification-patterns.md.” The first says what an agent file
looks like; the second says how to pick its verification shape. Everything else about a system
— “Each system's operational detail lives with the system”
— lives beside it, in its README, its command file, and its agent files.
The conventions doc is documentation, not runtime input: “This file is documentation, not runtime input. No agent reads it.” Every prompt is self-contained — “Every agent prompt is self-contained.” — and the duplication is deliberate: “A self-contained prompt cannot lose its guardrails because a shared file moved or was renamed.” What went wrong before the doc existed was not that the text repeated but that the repeats disagreed, so “The fix is canonical wording, not indirection.”
One section order, for every agent, under the heading “Structure — all agents, in
this order”: ## Boundary, ## Input, ## Missing input,
## Process, ## Output, ## Return to orchestrator. The
frontmatter carries name, description, and
tools: <exactly the tools the body uses. No more.>.
“Every agent is exactly one. Use the canonical sentence verbatim.” The sentence is copied into the agent file — not referenced from it.
| Class | The canonical sentence | In the corpus |
|---|---|---|
| A — observe only | “Read-only. You read remote data and write no files. You return a block to the orchestrator…” | calendar-briefing carries it verbatim. Its grant holds two Calendar tools
plus WebSearch, WebFetch — no write tool of any kind. |
| B — read remote, write local | “Read-only against every remote system. You write files inside this repo
only, and only the files named in ## Output.” |
The bulk of the corpus. Collectors, drafters, checkers and views all open with this line. |
| C — read remote, stage remote | “You may ONLY read messages and CREATE DRAFTS. You must NEVER send, reply-send, delete, trash,…” | email-triage, and only it. Its whole grant is
search_gmail_messages, get_gmail_message_content, draft_gmail_message. |
One agent carries no class sentence at all. paper-ingester files
PDFs into a library outside this repo, so “write files inside this repo only”
would be false of it. Its boundary narrows the tool instead: “Bash is granted
for exactly two purposes”, and “WebFetch is granted for exactly one
purpose”.
One shape, three states, in JSON for agents that write JSON:
{"status": "ok" | "missing" | "unconfigured", "period": "<YYYY-MM>", ...}.
missing is “a required input file was absent or
unreadable”; unconfigured is a config file not yet filled in — “The
user has not set this up yet. This is not an error; it is a normal first-run state.”
| Rule | Why it exists |
|---|---|
“Never fabricate to fill a gap. A missing input produces
missing, never a guess.” |
A guess is indistinguishable from a figure on the deliverable the human files. |
“An empty array is not missing. [] is a valid answer
(a month with no invoices).” |
The reader must be able to tell “checked, found none” from “did not check”. |
| “Never compute from partial data. Downstream agents cannot tell a partial result from a” complete one. | /nai-tax-theo applies it to a three-month rollup: with any month absent it
does “not compute a partial quarterly sum”. |
“Every agent has a ## Missing input section. No
exceptions.” |
The doc names the agent that shipped without one:
“vat-worksheet shipped without” it — which now says
“A worksheet built on partial data looks identical to a correct one.” |
“Every path is repo-root-relative and written in full. Always.”
because “A subagent has no reliable notion of a ‘current period folder’ —
the period is a parameter it” was handed. The canonical-phrasings table
settles the small things that used to drift: the dispatch tool is “Agent” — never
“Task”; the boundary heading is ## Boundary — never
## HARD SAFETY BOUNDARY, ## Hard boundary, or
## Guardrails.
And no phase numbers in a description, because two numbering schemes once disagreed: “The orchestrator's wave model is the truth.” — “Agents describe what they do, not when they run — they do not know when they run.”
“Grant exactly what the body uses. Not one tool more.” This is the
convention with a scar: “Read, Write, Glob, Grep was once pasted as a default
suffix onto 6 of 10 agents”, and two of those grants contradicted the agent's own stated
boundary. Hence the rule and its check:
“A grant that contradicts a boundary is worse than a missing grant. A missing grant fails
loudly.” — docs/agent-conventions.md. The check is mechanical:
“Check before every commit: for each tool in tools:, find
where the body uses it. If you” cannot, remove it. CLAUDE.md lists the same
step under Verifying changes: “Check each agent's tools: frontmatter against
what its body actually uses.”
docs/verification-patterns.md is the second build-time document, and its
“Choosing a shape for a new system” section is a decision procedure, asked in order:
The through-line is stated up front: “verification rigor should match how expensive a wrong answer is”. And a system that outgrows its shape gets moved, not patched — if it later starts “feeding automated decisions — that's a signal to move it up a shape, not to bolt” verification onto its existing prompt. The four shapes themselves are walked through on the verification page.
Because prompts are self-contained, the same four shapes of agent recur across systems, each carrying its own copy of the rule that defines it. That repetition is the convention working.
| Archetype | Instances | The rule it carries |
|---|---|---|
| Scoper | ea-context-reviewer, review-scoper,
spec-context-scoper, doc-scoper |
Runs once, inventories the drop, and writes the brief everything downstream builds on. “You open an architecture review: you inventory what was dropped for review, establish” … / “You open a spec draft: you inventory the dropped model instances…” |
| Drafter | finding-drafter, spec-rule-drafter |
No quote, no output. “Concerns without citable evidence become open questions, never findings. No web tools by design.” / “Patterns without citable evidence become open questions, never rules. No web tools by design.” |
| Checker | finding-checker, spec-rule-checker,
doc-checker |
Adversarial, and it never gets to be generous: “Default to UNSUPPORTED when uncertain.” — the identical sentence in two files — and “You never mark a claim CONFIRMED because it is obviously true.” |
| View | cockpit-writer, doc-renderer |
Renders, never computes or improves. “You are a view, never a calculator.” / “You never author, edit, shorten, expand, correct, or reorder the author's prose.” |
Two agents hold Bash, and both narrow it inside the prompt rather than relying on
the tool name. qonto-collector: “Bash is granted for ONE”
purpose only, curl-ing receipts. paper-ingester goes one step further and puts the
guard below the prompt: its download runs
curl -sSL -f -o <temp-path> and rejects anything that doesn't start with
%PDF — “(a paywall/error page saved as if it were the PDF)” —
and its boundary notes that “This isn't prose-only: .claude/settings.json denies
any mv/cp/curl Bash command” touching the wiki, at the
permission layer.
CLAUDE.md counts the corpus plainly: “Twelve command files, ten
systems.” Two of the twelve are deliberately not systems — “Two commands in
.claude/commands/ are not agent systems and are deliberately absent
from” the list. The same conventions stretch across all of them:
| Command | What it shows a builder |
|---|---|
/nai-payroll-theodora |
The smallest shape: one dispatch, one synthesis, and an explicit non-feature — “No cockpit. This is a small monthly utility run, not a CFO-facing review package — CFO” sign-off happens elsewhere. |
/nai-option-study |
The bound stated as absolute: “You never run a fourth round. The bound is” absolute. And the interface discipline: the recommender gets the round number, “never the critique text; the files are the interface.” |
/nai-review, /nai-spec-drafter |
The deliberate contrast, written twice: “one adversarial verify pass, no
loop.” / “Like /nai-review, this system verifies with
one adversarial pass, no loop.” |
/nai-newsletter |
Scaffolding that never destroys work — “They may have hand-edited the draft. Never silently overwrite.” — and an honest exit: “A quiet month is a valid, honest outcome. Never pad.” |
/nai-issue-to-agent |
A STOP in front of an irreversible remote action: “STOP and wait for the user's explicit confirmation” before posting a comment that cannot be deleted. |
/nai-issue-check |
Failure is never quiet: “a failed check is not ‘nothing happened,’ it's unknown, and should not be” silently treated as quiet. |
/nai-paper-ingest |
Gate before you dispatch: “No candidate found → STOP.” — no subagent, no comment. |
/nai-cohort-census |
A shareability boundary: “Never include file contents, prompt text, or any data an agent has touched — names” and counts only. |
/nai-document |
Staging over publishing, and no invented paths: “Page names do not derive from command names. This table is the truth; never construct a filename” from a slug. It “never authors prose”, and “Promote automatically only if all three hold:” — otherwise the page stays staged. |
The one agent five systems share is the view: “cockpit-writer is shared by
five of the ten systems”, and it “is a view only; it computes nothing and every figure
it shows appears verbatim” in the source markdown.
A guardrail you can only find by following a reference is a guardrail that can go missing. So every prompt repeats its own boundary, its own missing-data contract, and its own tool narrowing — and the conventions doc's job is to keep those repeats identical rather than to remove them.
The three systems were built over five sessions. They then took one more session, of about the same size, to make trustworthy — and that session shipped no new features at all.
| Building the functionality | Hardening it | |
|---|---|---|
| When | 5 – 14 Jul, five sessions | 14 – 15 Jul, one session |
| Commits | 54 | 27 |
| Measured active time | ~6.4 h | ~4.7 h |
| What came out | 3 commands, 10 subagents across those three systems, 3 design specs, 3 plans — all three systems shipped and ran for real | Zero new features. 6 correctness fixes, 1 confidentiality leak closed, 10 agent prompts standardized, 1 misleading doc deleted |
Hardening cost roughly 73% of what building cost. Not because the build was sloppy — every one of the three systems worked, shipped, and had produced real output. The tax pipeline had run a real month-end close. Hardening did not fix broken. It fixed looked right.
| Dispatches | Agent compute | Tokens | |
|---|---|---|---|
| Doing the work | 20 | 2.4 h | 1,789k |
| Checking the work | 13 | 0.6 h | 1,027k |
| Total | 33 | 3.0 h | 2,816k |
39% of the dispatches were verification — agents whose only job was to attack work another agent had just declared finished. That is the line item most likely to look like waste on a budget, and it is the one that paid. The reviews caught a guard that failed open, three rules silently deleted during a “style” pass, a VAT rule that was factually wrong about German tax law, and — at the very last gate, after every individual task had already been reviewed and passed — two more defects, one of which would have silently overstated a VAT return.
10 of the 27 commits were corrective: fixing work done earlier in the same session. Including two of my own unforced errors — a plan that quoted the client's confidential data verbatim into a new tracked file while describing how to remove it, and a history rewrite that missed a second copy of the same file under an older path.
Read these numbers honestly. They are wall-clock between commits, not effort: gaps, reading and thinking are invisible. The build's 6.4 h undercounts — design and brainstorming before a day's first commit leave no trace. The human's time is in neither column. Agent compute (3.0 h) overlaps wall-clock rather than adding to it, since dispatches ran in parallel. Treat the ratio as the finding, not the absolute figures. They measure one July 2026 window and three systems — not the workspace as it stands today.
The useful thing to take from this: when planning agent work, the build is roughly half the job. An agent that runs is not an agent you can rely on, and the distance between those two is about as much work as getting it to run in the first place.
NextRunning GSD on a Web Application →