When someone shows me an AI system, I don’t start by asking about the model or the platform. I ask four questions. I’ve asked them so many times by now that I’ve started calling them the Kiil principles for AI.
They sound simple, and they are. Still, it’s surprising how few people can give a sharp answer. I’ve had to rebuild my own systems more than once because they couldn’t either.
1. Which parts are rules, and which are model? #
A language model is good at judgement. It can read a sloppy email, summarise a report and suggest a phrasing. It’s bad at rules. It can’t be trusted to calculate VAT, keep a deadline or remember that a customer in arrears doesn’t get credit.
Rules belong in code that runs the same way every time and can be tested. If your prompt says “always remember to”, you’ve written down a wish, and nobody can test a wish.
In my own job application system the model may write the cover letter freely. In the CV it may only cut. Every line it proposes is compared word for word with my master CV, and if a single one differs, the CV isn’t written at all. So the model can’t invent experience I don’t have. The prompt asks it not to, but it’s the code that makes sure.
2. What happens when the model gets it wrong? #
Because it will. The only question is whether anyone notices.
The dangerous failures are the quiet ones. A model that doesn’t know the answer answers anyway. A broken integration reports “no new data”, and that looks exactly like a slow day. I once had a data source return zero results for four days without anything raising a flag.
So validate before you write. Stop the process rather than return an empty answer that looks like a real one. And remember that a check you’ve only ever seen pass is a check you don’t know works. You need to have seen it fail at least once.
One of the best tricks I know is to ask the same question twice: once with rules and once with a model. Where they agree, the case moves on. Where they disagree, a human takes a look.
3. Can you show me afterwards why the system made a particular decision? #
I’m not asking what the model said. I’m asking why the system did what it did.
A log of prompts and responses is a transcript of a conversation. An audit trail follows the decision: which case it was, what data went in, which rule kicked in, what the outcome was, and who approved it.
It sounds like bureaucracy until a customer complains or the auditor calls. In my case, the trail is what revealed that an automatic reminder was about to chase me for something I’d already done.
The EU AI Act requires logging for high-risk systems. But law or no law, management can’t take responsibility for something it can’t see.
4. Which actions require a human, and why those in particular? #
“A human approves everything” lasts about three weeks. After that the approval is a click nobody reads, and you have a rubber stamp with a log file behind it.
Humans belong in two places: where the system knows it’s unsure, and in front of actions that can’t be undone.
My test is a single question. Can it be reversed? A draft on my own disk I can simply make again, so that runs on its own. An application that has been sent can’t be called back, so there the system waits for me. How hard a step is doesn’t decide it.
The platform comes later #
Notice that none of the four questions is about which model, cloud or vector database the solution runs on. That has to work too, of course, but it’s an operations job, and it can be solved.
If someone can answer the four questions sharply, the rest is detail. If they can’t, no platform will save them.
Take them with you the next time you buy, hire or build something with AI. And feel free to ask them of me.
I’ve written a longer version with more examples (in Danish) on docujai.com: Platformen er ikke arkitekturen. If you’d like help asking these questions of your own systems, send me an email.