Rules in code, judgement in the model

The first of the Kiil principles for AI is a question.

Which parts are rules, and which parts are model?

I ask that one first. The answer decides the shape of everything else. Get it wrong, and no amount of prompt tuning will save you.

Two kinds of work

#

Almost every task you’d hand to an AI system contains two kinds of work.

Some of it is judgement. Which of these ten projects is most relevant to the job ad? Is this email angry or just short? What’s a better way to phrase this paragraph? There’s no single right answer, and a language model is genuinely good at this.

Some of it is rules. The VAT is 25 percent. The deadline is Friday. A customer in arrears doesn’t get credit. My CV must not contain experience I don’t have. There is a right answer, and it’s the same every time.

A language model can do the first kind. It cannot be trusted with the second. Not because it is stupid, but because it produces plausible text. Plausible is not the same as correct. It will follow your rule 98 out of 100 times. You will not know which two times it did not.

“Always remember to” is a wish

#

You can spot rules hiding in the wrong place by reading the prompt. Look for phrases like these:

• “Always remember to …”
• “Never …”
• “Make sure that …”
• “Do not under any circumstances …”

Each one is a rule written as a request. The model will usually honour it. But you can’t test it, you can’t prove it held, and you won’t see the day it breaks.

The fix is to keep the request in the prompt, because it helps, and then enforce the rule in code afterwards. The prompt asks. The code makes sure.

An example: a CV the model may only cut

#

I have a system that tailors my CV to job ads. The model gets my master CV and the ad, and picks the lines that fit. That’s judgement, and it does it well.

The rule is that it may only cut. It must not rephrase a line into something slightly grander, and it must not add anything. In the prompt I ask it nicely. In the code I check every line it returns against the master CV, word for word.

Here is a simplified version in Nushell. I use aichat for the model call, but any command line LLM tool will do.

# Rule: the model may only cut. Every proposed line must exist verbatim in the master CV.
export def assert-cut-only [master: list<string>]: list<string> -> list<string> {
  let proposed = $in
  let invented = $proposed | where $it not-in $master
  if ($invented | is-not-empty) {
    error make {
      msg: $"The model proposed ($invented | length) line\(s\) that are not in the master CV"
      help: ($invented | str join (char newline))
    }
  }
  # Keep the master order, whatever order the model answered in
  $master | where $it in $proposed
}

# Judgement: let the model pick the lines that fit the job ad
export def propose-lines [master: list<string>, job_ad: string]: nothing -> list<string> {
  let prompt = [
    "Below is my CV, one item per line, followed by a job ad."
    "Return the lines from the CV that are most relevant to the job."
    "Copy each line exactly as written. Do not rewrite, merge or add anything."
    "Return only the lines, one per line."
    ""
    "CV:"
    ...$master
    ""
    "Job ad:"
    $job_ad
  ] | str join (char newline)

  ^aichat $prompt | lines | str trim | compact --empty
}

export def tailor-cv [master_file: path, job_ad_file: path]: nothing -> list<string> {
  let master = open --raw $master_file | lines | str trim | compact --empty
  propose-lines $master (open --raw $job_ad_file) | assert-cut-only $master
}

Save it as cv.nu and run it like this:

use cv.nu tailor-cv
tailor-cv master-cv.txt job-ad.txt | save tailored-cv.txt

Notice how the work is split. propose-lines is the only place the model appears, and it only makes a choice. assert-cut-only contains no AI at all. It’s a plain function that takes a list and either returns it or stops.

Two details matter.

First, when the rule is broken, the command fails with an error. It doesn’t quietly drop the bad line and carry on. If the model starts inventing things, I want to know, because it probably means the prompt or the model has changed under me.

Second, the order of the CV is decided by the code, not by the model. That’s a rule too. It’s a small one, but there’s no reason to leave it to chance.

Rules can be tested

#

Because the rule lives in a plain function, I can test it without calling a model at all. That makes the tests fast, free and the same every time.

use std/assert
use cv.nu assert-cut-only

let master = [
  "Built a Nushell module for Drupal publishing"
  "Ran workshops on AI for teachers"
  "Wrote a book on prompting"
]

# A pure cut passes, and comes back in master order
assert equal (
  ["Wrote a book on prompting" "Built a Nushell module for Drupal publishing"]
  | assert-cut-only $master
) ["Built a Nushell module for Drupal publishing" "Wrote a book on prompting"]

# Invented experience fails
assert error { ["Led a team of 40 engineers"] | assert-cut-only $master }

# So does a line that is only almost the same
assert error { ["Ran workshops on AI for teachers."] | assert-cut-only $master }

The last test is the one I like best. A single full stop is enough to fail. That’s harsh, and it’s meant to be. A model that adds a full stop today may add “senior” tomorrow, and I’d rather catch the full stop.

The tests also show the check failing, not just passing. That matters more than it sounds, and it’s really the second principle, so I’ll save it for the next post.

Where to draw the line

#

You won’t always find the split as clean as in the CV example. A few questions help me:

• Is there a right answer? If two careful people would always agree, it’s a rule. Write it in code.
• Could I write a test for it? If yes, write the test, and let the code pass it, not the model.
• What does it cost if it’s wrong one time in fifty? If the answer is “a slightly clumsy sentence”, let the model do it. If it’s “a false claim on my CV”, don’t.

And often the best design is both: the model proposes, the code checks. The model is free to be creative inside a fence, and the code is the fence.

The model can change. The rules can’t.

#

There is one more reason to keep rules out of the prompt. Models change. A new version arrives, a provider tweaks something, you switch to a cheaper model to save money. Your prompt may suddenly work a little differently.

If your rules live in code, that doesn’t matter. The model can get better or worse at choosing CV lines, and my CV will still only contain things I’ve actually done.

That’s what the first principle buys you. Not a smarter system, but one where you know exactly which parts you can trust.

If you’d like help finding the rules hiding in your own prompts, send me an email.