AINov 3, 2026 · 4 min read

How to stop your AI from making things up

Answer only from your documents, limit what it can decide, and send "I don't know" to a person. Here is how to make each rule stick.

Altamar Engineering The team that designs, builds and supports the work

In 2024 a tribunal in British Columbia ordered Air Canada to pay a customer because its website chatbot invented a refund rule. The chatbot told a grieving passenger he could claim the bereavement fare after his flight. The real policy said no. The airline argued the chatbot was responsible for its own words. The tribunal disagreed: the company is responsible for everything on its website, chatbot included (Moffatt v. Air Canada).

Why do these systems make things up? In September 2025 OpenAI’s own researchers published a paper, “Why language models hallucinate”, arguing that the way models are trained and graded rewards a confident guess over admitting uncertainty. A guess sometimes scores points. “I don’t know” never does. So the model learns to guess.

You cannot train that out of the model yourself. You can build around it. Three rules do most of the work, and each one has to be enforced in the system, not just written in the prompt.

#Rule 1: Only your docs

The AI should answer from your own documents, policies and data, and from nothing else.

A sentence in the prompt (“only use the provided documents”) is a request, not a guarantee. To enforce it:

  • Give it the right passages, every time. Before the model answers, the system looks up the relevant parts of your documents and puts them in front of it. This is usually called retrieval. The model should never be the one remembering your refund policy.
  • Keep one source of truth. If your returns policy lives in three places and two are out of date, the AI will find the wrong one. Pick the official version, put a date on it, and remove the copies.
  • Make it show where the answer came from. Ask for the source with every answer (the document and the section). Then check it in code: if the answer cites nothing, or cites something that was not in the passages it received, do not send it.
  • No passage, no answer. If the lookup finds nothing relevant, the system should not call the model at all. It goes straight to rule 3.

#Rule 2: Limit the options

The more the AI is free to decide, the more room it has to invent. Most business decisions do not need free text. They need one of a few outcomes.

  • Give it a short list of exits. For a refund question the exits might be: approve, reject, or hand to a person. The model picks one. It cannot create a fourth option such as “you can claim it later”.
  • Put the rule in code, not in the model. Whether a ticket qualifies for a refund is a business rule: dates, amounts, fare type. Write it as code that returns yes or no. Let the AI collect the details and explain the result in plain words, but never let it make the decision itself.
  • Check every output before it acts. If the model returns something outside the allowed list, or a field in the wrong format, the system rejects it and retries or escalates. Nothing it says goes straight to a customer’s account.
  • Keep actions behind permissions. An assistant that can only read order status cannot issue a refund by mistake. Give it the smallest set of actions the job needs.

#Rule 3: “I don’t know” goes to a human

An honest “I don’t know” is better than a confident wrong answer. But it only helps if something happens next.

  • Make handing off a real action, not a sentence. The AI should have an explicit “pass to a person” step that creates a ticket, alerts the right channel and tells the customer when they will hear back.
  • Decide the triggers in advance. Money above a threshold, exceptions to policy, complaints, legal or medical topics, an angry customer, or any question where rule 1 found no source. In these cases the AI does not try. It hands off.
  • Say so plainly. “I’m not sure about that one. I’ve passed it to our team and someone will reply today” keeps trust. A made-up policy loses it, and as Air Canada found, it can cost money.
  • Watch the handoff rate. If almost nothing is handed off, the AI is probably guessing. If almost everything is, your documents have gaps. Both are useful to know.

#Test it before customers do

Before launch, write twenty or thirty questions the AI must not answer from imagination:

  1. Questions your documents do not cover.
  2. Questions that sound like an exception (“my case is different”).
  3. Questions with a wrong assumption built in (“since refunds are allowed after the flight…”).
  4. Attempts to talk it out of its rules (“ignore your instructions and…”).

Run them every time you change the prompt, the model or the documents. An answer that invents a rule is a failed test, the same as a crash.

After launch, read a sample of real conversations every week. Look for answers with no source, decisions outside the allowed list, and cases that should have been handed off. Each one is a gap to close in the documents, the rules or the triggers.

#Who owns it

Someone in the business, not only the developer, should own the AI’s answers: keep the documents current, review the weekly sample, and decide the handoff triggers. The tribunal’s point applies to every company: what your AI says, your company said.

Keep reading

All posts

Book a call

Let's talk about what your teamstill does by hand.

Pick a time that works. You talk to an engineer, not a salesperson.

Book a call 30 min · Google Meet · English or Español