AI Doesn't Bring Judgment. It Borrows Yours.

"The higher the value of the work you point AI at, the more judgment it needs. And AI has none of its own. It borrows the judgment of whoever is guiding it. Which means the person beside it has to have some to lend."

How many times has AI handed your team something that looked right, sounded right, and was almost right, and nearly went out the door before anyone caught it?

That's the version of "AI gone wrong" that never makes headlines. It's not a chatbot saying something offensive. It's a quote that missed the one clause a client always cares about. An onboarding step that missed the PO number so you can get paid on time. A contract review that flagged a typo and waved through the actual risk.

I watched this play out with a business owner a few weeks back. He'd bought an AI automation that looked impressive on the surface. It handled a piece of work his team hated doing manually, the automation supplier had scoped it properly, the workflow looked sensible and the demo worked.

It was the real work, once it started coming through, where things went sideways.

The output wasn't broken enough to reject, that would have been easy. It was worse than that: plausible, confident, almost right.

None of the mistakes came down to the tool, and it wasn't the supplier's fault either. What was missing was judgment - specifically, someone close enough to the work to catch what the AI couldn't.

This is a pattern I see often. A business decides it's time to get serious about AI, so it does what it's always done with technology: calls in a supplier, usually an IT company that now does AI, scopes a few automations, and waits for them to work.

It's an understandable move, and it's also the trap, it's how they've always bought technology. Scope it, install it, tick it off.

That's the promise, anyway. The reality of IT projects is more familiar: they can run long, cost more, and deliver less than the pitch. It's where a lot of the scepticism about AI already comes from. Approach AI the same way, and you'll prove the sceptics right.

AI doesn't work like an installation, though, and the higher the value of the task you point it at, the more obviously that becomes true.

The automations get built. Something with teeth, because that's where the money is. Quoting. Client onboarding. A first-pass review of proposals or contracts. The work that involves judgment, which is exactly the work worth automating, and exactly the work that goes plausibly, confidently wrong when no one capable is watching.

So a review step gets added. But the person reviewing often can't tell what's wrong either, because they were never operating at the level the task demanded. The automation was pointed at judgment-heavy work, and no one in the loop had enough judgment to lend it.

The project stalls. Everyone concludes AI was overhyped, when what actually went wrong is that AI has no judgment of its own. It's extraordinary at producing plausible work at speed, but the judgment in that work is borrowed. It comes from the person guiding it: the way they frame the task, the standard they hold it to, the errors they catch, the context they feed in, the corrections they make until the output is right. Take that person away, or put someone below the level the work demands in their place, and the judgment simply isn't there. The machine doesn't supply it. It never did.

The higher the value of the work, the more judgment it needs, and the more capable the person beside it has to be.

Which is why this can't be bought, only built, and why treating it like an IT rollout is the category error underneath the whole thing. You can install software in a week. You cannot install AI capability into a person. Capability is grown, one level at a time.

We map that growth in five levels, from someone who occasionally asks AI a question, up to someone who builds and runs their own AI systems and thinks alongside them. The levels matter less than the one rule that governs all of them: you move up by changing your routine and using it until the new way of working sticks, the way you learned to drive. The tools gain new powers every month. People climb at human speed. There's no sped-up version you can purchase.

Climbing is precisely the process of learning to lend your judgment to the machine. It's someone who can take what they know, the standard they hold, the things they'd never let slip, and get the AI to hold that standard too. That's the skill the levels measure, and it's the exact thing the bought-in automation was missing.

So when a business runs ahead and points AI at high-judgment work with no one capable enough to steer it, one of two things happens.

1. The project fails slowly but surely. The output can't be trusted, no one can say precisely why, and it gets shelved.

2. Or it works, but only because the outside supplier is the one running it. They're operating at the level your people aren't. The automation delivers, but you're now permanently dependent on them, renting a capability you were meant to own, exposed the day they raise their price or move on.

Every automation that touches judgment-heavy work does one of two things. It either builds capability inside your business, or it moves dependency outside it.

That might get the project live. But it doesn't build the capability you thought you were buying.

The fix isn't to slow down on AI. It's to stop pointing it at work your people can't yet steer, and to build the capability that lets them steer it.

Start with the question most businesses skip. Not "which supplier do we call?" but "what level are our people actually at?" Then match the work to it.

Low-judgment, fast-feedback work is the place to start: drafting the first version of a job ad, summarising a long email thread before a meeting, cleaning up scattered notes into action items. If AI gets it 80% right, someone catches the other 20% in seconds, and nothing bad happens. It pays off almost immediately, with almost no risk.

High-judgment work is different: the quote that sets the tone for a client relationship, the first-pass contract review that decides what a lawyer actually sees, the onboarding sequence that either catches or misses the thing that makes this client different from the last one. Get 80% right, and the 20% that's wrong is the part that costs you the client. It's the part a junior reviewer won't catch, because they don't yet know what "wrong" looks like in that context. Save this work for where you have someone able to lend the judgment, and grow more of those people on purpose, because they're the ones who unlock everything above them.

Match the ambition to the capability. Get it right and AI compounds. Get it wrong and you've paid for a failed project, or a landlord.

AI borrows its judgment from whoever is holding the reins. Make sure someone is.

Find the risks your AI cannot catch on its own

AI does not fail loudly. The real risk is work that looks right, sounds right, and slips through before anyone with the right judgment checks it.

Take the 5-Minute AI Readiness Snapshot

--

About the Author 

Mark Lucas is Co-founder at Generation AI, where we help New Zealand business leaders build real AI capability.

Information provided is general in nature and does not constitute legal advice