Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages

Help for owners

My AI support agent gives customers wrong answers

Short answer

Models can produce text that is wrong or inconsistent with the context given, even the best ones. Reduce it by feeding the agent your real documents, allowing it to say it does not know, requiring a supporting quote for each claim, and testing against real questions. Mistakes drop but never reach zero, so risky topics need a person.

  • Written for business owners
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Does this sound like your situation?

  • It told a customer a refund policy I do not have.
  • The answers sound confident but are false.
  • Different answers each time for the same question.
  • It quotes old prices.
  • Staff no longer trust the tool.

Why does this happen?

  1. The agent is guessing from general knowledge

    Anthropic suggests explicitly restricting the model to the information provided and not its general knowledge. Without that it fills gaps with plausible text.

  2. It has no permission to say it does not know

    Anthropic says giving the model explicit permission to admit uncertainty can drastically reduce false information.

  3. Your documents are old or contradict each other

    The agent can only be as accurate as the material it reads. Two versions of a policy produce two answers.

  4. Nothing checks the answer

    Anthropic describes having the model cite a supporting quote for each claim and retract claims it cannot support. Without such a step, errors reach customers directly.

What can I do about it today?

  1. Collect the wrong answersSave the exact question, the answer and the correct answer for each one.
  2. Pause the risky topicsSwitch refunds, pricing and legal questions to a person until fixed.
  3. Fix the source documentsRemove outdated pages and keep one current version.
  4. Tell customers it is automatedCustomers should know they are talking to a bot and how to reach a person.
  5. Do not just retype the promptChanging wording without testing may fix one case and break another.

When should I bring someone in?

  • Wrong answers involve money, safety, health or legal commitments.
  • You cannot reproduce the failures.
  • The agent is already live with many customers.

How do I handle it?

  1. Build a test setThe wrong answers become a fixed set of questions with correct answers.
  2. Limit the agent to your sourcesIt answers only from approved documents and says when it cannot find the answer.
  3. Require support for claimsAnswers cite the passage they came from and unsupported claims are removed.
  4. Add a person for riskDefined topics always go to a human.
  5. Re-test on every changeThe set is run after each change to documents or instructions.

The price depends on what I find, so I quote in writing after a short discovery call. Tell me what is happening and I reply within 24 hours.

What do owners ask about this?

Can wrong answers be removed completely?

No. Anthropic says its techniques significantly reduce hallucinations but do not eliminate them, and advises validating critical information.

Should I switch to a bigger model?

It may help but does not replace good sources and checks, so test before changing.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Found a mistake? Tell me.

For your developer: the technical fixes

Next step

Want this off your desk?

Send me a few lines on where things stand. I reply within 24 hours.