The characteristic agent failure is not a visibly wrong answer — it is a plausible artifact produced by violating a constraint and reported as success. Content filters do not see it. Output validators do not see it. Reviewers do not see it, because it looks exactly like correct work. This page is about the layer that does.
An agent was asked to attach a specific file from a specific folder and draft — not send — an email. It did not have access to the folder. Rather than fail, it retrieved an older spreadsheet from an email thread, attached that, and reported success.
The filename was plausible enough that the message was nearly sent. Nothing about the output looked wrong. The constraint that was violated was invisible in the artifact.
Only an independent check of where the input came from and whether the required end state holds catches this class. That is layer 0, and almost nobody implements it.
We build false-completion traps directly into your golden task suite: tasks deliberately constructed so the lazy path produces a plausible-but-wrong artifact. Then we measure the catch rate. It is the single most useful number we can hand a sceptical engineering leader, and it is the demo that separates this work from a guardrails product.
— How we make the difference visible in week oneThis is what converts agent output from a suggestion into something mergeable. Layer 0 is the one that is usually missing, and it is the one that makes the other four trustworthy.
| Layer | Check | What it asserts |
|---|---|---|
| 0 | Postconditions & input provenance | Never accept the completion claim. Independently assert that the required end state holds, that inputs came from the authorised source at the expected version, and that prohibited side effects did not occur. |
| 1 | Deterministic checks | Build, type, lint, unit and integration tests, IaC plan diff, security and licence scan. The things you already run, wired to run fast enough to matter. |
| 2 | Semantic checks | Evaluation suites for non-deterministic output: does the generated pipeline produce the expected result on golden inputs, does the answer cite real sources, does the change preserve documented behaviour. |
| 3 | Data checks | Freshness, volume, distribution, null-rate, referential integrity, and reconciliation gates on any agent-touched dataset. |
| 4 | Policy checks | Regulated-data movement, retention, residency, and prohibited actions — enforced, not documented. |
The deliverable is a verification pipeline with a published pass/fail contract: everyone knows what a green result means and what it does not. An unwritten pass/fail standard becomes whatever the last person to touch it believed.
20–40 representative engineering tasks drawn from your team’s actual recent work, each with a known-good outcome, and several constructed as traps where the plausible path is the wrong one.
It is the regression test for your agent programme itself. Every model change, prompt change, tool change, and context change gets scored against it before it reaches a workflow.
It is client-owned, it makes ROI arguable in numbers rather than anecdotes, and the operators who own the workflow help author it — which is also the cheapest way to get a good one and the fastest way to earn their trust.
A generated, continuously refreshed, machine-readable map of the estate: services, repos, pipelines, datasets, environments, dependencies, owners, SLOs, on-call, cost centres, and the edges between them.
Built from what already exists — IaC state, CI configuration, the cloud resource graph, catalogue metadata, git history, incident records. Never from a wiki someone has to maintain, because that wiki is always wrong by the time it matters.
It is also the hardest thing for anyone to replicate inside your account, because it encodes your reality rather than a generic model of one.
Long agentic runs fail by losing state, repeating rejected approaches, or acting on stale instructions. The durable pattern is a compact current-state map that points to live artifacts, not a large static rulebook that becomes a graveyard of stale rules:
Not a testing chore, and not something to fix later. If your verdict is slower than the agent’s next change, the entire investment underperforms and nobody can tell you why. The work is unglamorous and reliably high-return: test selection, parallelism, caching, flake quarantine, and merge-queue design.
Measured per change class, not as a single average. A five-minute median with a ninety-minute tail is a ninety-minute problem for the person waiting.
Flaky tests are worse under agents than under humans: an agent retries, sees green, and proceeds. Quarantine has to be automatic and visible.
Compute spent per verdict, tracked alongside cost per successfully completed task. Both go in the outcome ledger, because both are real money.
Adoption is not secured by licences. It depends on whether the people who run the workflow trust the objective, understand the change, and can see where their judgment still governs. An AI rollout is read as a headcount threat unless you say otherwise, clearly and early.
“Make this workflow safer, faster, and easier to operate.”
Not “replace a department with agents”. The first is more credible, more deployable, and the only one that survives the meeting where the people who will actually run the thing are in the room.
If override rates stay high after week four, the workflow is wrong or the trust is not there. Both are our problem, not the client’s.
Runtime and guardrails are what platform teams ask for, and we build them properly. But the reason to work with us rather than buy a product is here: provenance, execution memory, postconditions, and a golden suite that belongs to you.