5 Ways to Check Whether an AI Answer Is Right: Sources, Re-asking, Number Checks, Cross-Checks, and Marking the Unconfirmed

Why does AI get things wrong so convincingly?

You asked an AI to write a notice for a client, and the result looks almost too smooth.
The dates and numbers are all there, and nothing sounds off.
Right before you hit send, a thought flashes by: "Is this actually right?" and your hand stops.
You may have been there.

That worry is well founded.
A chatbot answers a question by generating text that is likely to fit the question and context.
It does not look a fact up in a stored dictionary, so whether it knows something or not, it answers in the same confident tone.
People often call it "hallucination" when an AI fills a gap with something plausible.
The rate differs by service, but no service promises it will never be wrong.

That doesn't mean you should stop trusting AI. The aim is to spend less time sorting right from wrong.
Let's start with Method 1.

Method 1: Can you ask for a source and open it yourself?

If you ask an AI, "Tell me the source for this," it gives you a URL or a document name.
The catch is that you don't know whether that address really exists.
It can invent one, so a source only means something once you open it yourself.

We first asked: "What does HTTP status code 418 mean?" Before checking anything, we wrote down the AI's answer.
It said the code means "I'm a teapot," that it comes from RFC 2324, an April Fools' document published on April 1, 1998, and that it is now listed as "unused" in the standard table.

The part worth checking was the "standard table." We opened the HTTP status code registry kept by the Internet Assigned Numbers Authority (IANA), found the 418 row, and it said "(Unused)." So we saw with our own eyes that the second half of the answer was right.
When we first drafted this post, though, we had not yet checked the date "April 1, 1998" in the original document.
We did not treat that date as fact until we saw the original text.
More on that in Method 5.

If an answer comes with no source, change the question.

Method 2: What if you ask the same question twice, in different words?

If you ask the same thing in different words and the answers differ, one of them may be made up.
It helps to ask in a new chat window, where the earlier conversation affects the answer less.

We tried it with a Python question.
First we asked, "What does round(2.5) return in Python?" Then we asked, "Does Python round numbers ending in .5 up, or to the nearest even number?" Both answers landed on the same conclusion: 2, rounding to the even number.

One thing we want to be clear about is that this only adds a little reassurance.
The same AI can repeat the same mistake, so this method is good at catching errors but cannot prove an answer is right. Treat it as a strong signal only when the answers split.

Method 3: How do you check numbers, dates, and names on their own?

If you start doubting a whole answer, you get tired fast.
It is much quicker to pick out the numbers, dates, and proper names and check only those.
That is where the costly mistakes tend to cluster.

The Python question was a good example because we could run the calculation.
The AI said round(2.5)=2, round(0.5)=0, round(1.5)=2, and round(2.675, 2)=2.67.
When we ran the code, we got exactly 2 0 2 2.67. The last value is the one people often expect to be 2.68.
The reason is that 2.675 is stored inside a computer as 2.67499999..., and we were impressed that the AI pointed out that reason too.

For numbers you can't calculate, find the same number in the original and compare by eye.
When you copy figures into a table, checking just the number cells against the source saves a lot of time.

Method 4: How do you cross-check with another AI or a search?

This means checking by a different route.
Ask another AI the same question, or search and compare with an official source.
AIs from the same company may have learned from the same material, so we suggest one check with an AI and one with a human-written original where you can.

In our test we asked, "What is the latest document that defines HTTP status codes, and when was it published?" The answer was "RFC 9110, June 2022." On the RFC Editor page, the title was "HTTP Semantics," the publication date was June 2022, and the status was Internet Standard.
We checked only those three points; we did not separately confirm that no newer document has replaced it.
We should say that this time we did not use a second AI. We only compared the answer with the official original.

Method 5: How do you mark an answer you can't verify?

For anything you could not confirm, it is safer to mark it "unconfirmed" than to delete it.
That keeps it from hardening into a fact the next time you reuse the same content.

The "April 1, 1998" in the 418 example was exactly this case.
In the draft we left it as Unconfirmed. Only during review, after we checked the header of the original RFC 2324 text at the RFC Editor and saw "1 April 1998," did we turn it into a fact.
Because it was marked, it was obvious what still needed opening.
This is the same idea as the information-grade labels we use in other posts on this site.

How many methods should you use?

Honestly, if you checked every answer all five ways, there would be little point in using AI. The rule is how much it would hurt if the answer were wrong.
If you are just brainstorming or polishing a sentence, Method 2 alone is enough.
For a work memo or an explanation of a concept, use Methods 2 and 3 together.
If money, a contract, or a document you will submit is involved, use all five.

The time column is an estimate, not a measured value.
If you don't even have time to check, leave that answer as "unconfirmed" and don't use it.
That is the cheapest method of all.

What we tried ourselves

While writing this post, we wrote three questions based on public information.
We had Claude (Sonnet 5.5) write its answers first, then compared each one using the methods above.
The result: all three matched what we checked (for the RFC question, the title, publication date, and status).
We did not try to force a wrong answer, and we could not reproduce one.
That is why this post has no scene where the AI gets it wrong.

The main finding was elsewhere.
Even knowing the answers were correct required opening sources and running code.
There are limits, too.
For Method 2, we asked again in the same chat window instead of a new one, which may have weakened the effect.
And we only had three questions.

What we think

• Checking is a division of labor, not distrust. The AI drafts quickly, and the person looks only at the places where an error would hurt.
• We think a rule for using fewer methods to match the risk is more useful than adding more methods.
• If you keep a record of checks even on days the AI was right, you build up a picture of which kinds of questions it gets wrong.

FAQ

Q. If newer AI models make fewer mistakes, can I skip checking their answers?

They may make fewer, but nothing is guaranteed.
In our test, three answers were right, which doesn't mean other questions will go the same way.

Q. What if the source link the AI gave me won't open?

Don't use that source.
Search for the same content directly on the official site, and if you can't find it, leave it as unconfirmed.

Q. Can I ask another AI, "Is the answer above right?"

It helps as a reference but it is not proof.
Do the final check with a human-written original or your own calculation.

Q. Can I check medical or legal questions this way?

This post is a general guide to checking.
For high-stakes matters such as health, law, taxes, or investing, check with a professional in that field or an official body.

Q. What is the minimum to do when I'm short on time?

Check just three things: numbers, dates, and names.
If you can't do even that, mark the answer unconfirmed.

This post was written with the help of AI tools and reviewed by the site editor before publishing. How we use AI

Sources