Evaluate AI coding agents
Understand task scope, autonomy and the evidence your reviewers need.
The difference
Guardrails give AI agents rules for security, architecture and code quality. Anvil Coder includes versioned rule packs in prompts. Executable checks can block or report violations; an instruction in a prompt alone does not prove compliance.
The usual way is the other way around: generate first, check afterwards. A linter, a scanner, a review — and whatever fails gets reworked. That works for individual files.
For a project that emerges from hundreds of parallel work steps, it stops working. A structural mistake that arises in step 12 has propagated by step 200. Finding it then is no longer rework but rebuilding — and the context in which it arose is long gone.
The invariants do not live in documentation a model might have read. They live in every prompt.
The five invariants above are an excerpt of the baseline — the part of the catalogue that applies to every stack, independent of language and framework. Beyond that, every project automatically gets the stack packs that fit its technology: a React/Vite project gets different rules than one on Spring Boot, without anyone entering that by hand. Every pack is versioned — which version a run already in flight sees never changes retroactively.
Curated for public release today are eight stacks, from the language-independent baseline to the most widely used frontend and backend frameworks:
Every rule carries a severity. A blocking rule halts the run the moment it is violated; an advisory one shows up as a note without stopping it. When a rule does not fit a particular project, it can be relaxed with a justification or — where the rule allows it — disabled for a limited time. Some rules are structurally protected and stay in force regardless.
Every organisation chooses between two update channels. Fast, the default, automatically picks up the newest version of every pack. Stable only picks up a new version once we have explicitly promoted it for that channel — the slower, more deliberate path.
The catalogue is signed with an Ed25519 key: every rule file carries a sha256 fingerprint in a signed manifest, and no catalogue change takes effect without a fresh signature. The rules that bind a run are exactly the approved ones — cryptographically evidenced, not merely promised.
The invariants that ship with the system are the baseline, not the limit. A team that wants to enforce its own architecture rules — a particular layering, a naming scheme, a ban on certain libraries — stores them as its own guardrails. They are injected exactly like the built-in ones. Whoever wants one of them additionally validated by machine stores it as a machine-readable rule; as mere prose it acts in the prompt, but not in validation.
In the hosted service the system also collects suggestions for new guardrails from live operation. None of them is applied by itself: the only way into the catalogue is a human approval.
This is the point where architecture governance stops being a document nobody reads and starts being a property of the system.
They do not replace review. A system that checks every step against the same invariants produces consistent code — not necessarily correct code. Whether the requirement itself is right, whether the domain slicing holds, whether the solution fits the problem: that remains a human decision. Anvil Coder is built so that it stays possible: which actions need an approval is something you configure per project — without a configured level, the order runs straight through.
How a rule moves from the wiki into the run — categories, the #RULE pattern and the proposal inbox — is the subject ofGuardrails for AI coding agents.
Understand task scope, autonomy and the evidence your reviewers need.
Follow an assignment through planning, changes and the review handover.
Turn requirements and acceptance criteria into actionable tickets.