Skip to content
Agrim Intelligence

Insights · What we learned

Plausible is not the same as what you meant

By Agrim · Published November 16, 2026 · 3-minute read

In short

Why does work written by AI so often look right and still turn out wrong?

Because AI produces what is plausible, and plausible is not always what you meant. The remedy is to settle how the work will be checked before it starts, so the result is measured against what was asked.

We learned this on our own products, and it changed how we build.

What went wrong?

When we first built software with AI, we did it by prompting it one piece at a time. Ask for a screen, get a screen. Ask for the code behind a form, get the code. Each piece arrived quickly, and each looked right.

The trouble was in what each piece did not know. Every one was written without knowledge of the rest. Asked to solve the same problem twice, the AI solved it a different way each time. Nothing was wrong with any single answer. Put together, they did not agree with one another, and the product broke.

That is what "plausible" means in practice. The AI gives you something that could reasonably be the answer. It does not give you the answer that fits everything else you have already decided, unless it has been told what that is.

What fixed most of it?

Two things.

First, we gave the AI a shared memory of the whole project. What had been decided, how each problem had already been solved, and why. A piece of work no longer started from nothing.

Second, we gave it a fixed process to follow. The same steps, in the same order, every time: say what is wanted, look at what exists, write a plan, have the plan challenged, do the work, have the work reviewed.

That fixed most of it. The pieces began to agree with each other, because they were all written against the same record.

Code on a screen

What was left?

What remained is harder, and it has nothing to do with how capable the AI is.

AI writes faster than anyone can read. A person reviewing its work falls behind almost at once. So by the time you notice a wrong turn, a great deal has been built on it. The earlier the mistake, the more sits on top of it, and the more has to come out.

Left alone, the work becomes a black box. It runs, and nobody on the team can say with confidence why it is built the way it is.

More reviewing does not solve this. Nobody can read that fast, and a team that tries simply becomes the slow part of the process.

What do we do now?

Our answer is tight control at the points that matter, and none of it relies on reading everything.

The plan is written and challenged before the work starts. A wrong turn caught in a plan costs minutes.

How the work will be checked is settled first. Before anything is built, we write down what "done" means and how anyone will know. The result is then measured against what was asked, not against how convincing it looks.

A person approves each step that matters. Not every line: each step where a decision is being made.

And "done" is checked, not claimed. In Sigma Loop, the open-source tool that came out of this, a piece of work is not marked done until the record shows that every phase ran and both reviews approved.

Is that not slower?

It is slower than letting the AI run. It is much faster than undoing what it built.

That trade is easy to misjudge. The cost of going fast is paid later, and by someone else: the person who has to find out why two parts of a system disagree. The cost of going carefully is paid at once, in plain view, and looks like delay.

We have paid both. Careful is faster.

What should you take from this?

If you are putting AI to work on anything that has to hold together, a codebase, a set of contracts, a body of test cases, three questions are worth asking.

Does it know what has already been decided? If each request starts from nothing, expect answers that disagree.

Was "right" written down before the work began? If not, you will end up judging the result by how plausible it looks, which is the one thing AI is always good at.

Who approved the plan? If nobody did, the first review of the idea is happening after it has been built.

None of this is a reason to hold back from AI. It is the difference between work that looks finished and work that is.

Evidence, not promises.

Bring us one process you would like to hand to AI. We will show you how it would be checked, and who would sign it off.