Skip to content
Defici
← Defici NewsAI News

Confidently Wrong

By Defici Editorial · 21 Sept 2026

AI-generated · Defici Editorial

Everybody who uses these tools for more than a week learns that they sometimes state things that are not true. What is less widely understood is the shape of the problem. The error does not arrive hedged, or vague, or flagged as uncertain. It arrives in the same measured, well-organised tone as everything else, frequently with supporting detail, and occasionally with a citation to something that does not exist. There is no tell. A person cannot detect it by reading carefully, which is the natural defence and which does not work here, because the text carries no signal of its own reliability.

Since the general warning is unactionable, the useful move is to know which categories of answer are risky, and they are reasonably predictable. Specific facts about the world - a date, a figure, a person's role, whether a company did something - are the highest risk, particularly when they sound like the kind of thing that ought to be true. Legal and regulatory specifics are high risk and consequential, and they vary by country in ways a general answer will smooth over. Anything recent is risky, since the tool may simply not know. Citations, references and quotations are notoriously unreliable and should be treated as leads to check rather than sources.

The lower-risk uses are equally predictable, and this is the more useful half of the picture because it says where checking is not needed. Rewriting something you wrote, summarising a document you provided, explaining a general concept, structuring an argument, drafting a first version, brainstorming, translating tone, finding the gaps in your own text - in all of these the material comes from you or from something you supplied, and the tool is operating on it rather than supplying facts. That is the great majority of what a small business actually uses these tools for, and it is comparatively safe, which is worth saying plainly.

So the practical rule is not to verify everything, which nobody will do, but to notice when an answer contains a claim about the world that you would not have been able to make yourself - and to check that one thing, in a source, before it goes anywhere that matters. The other discipline is to state the obvious once: if the answer is going into something a customer will rely on, a document that will be signed, or a decision involving real money, then a person's name is on it and the tool's is not. It has no stake in being right, and that is not a criticism of it - it is a description of what it is.

This article was generated by Defici's AI editorial system.

ShareXWhatsAppLinkedIn

Get Defici News in your inbox

Confidently Wrong - Defici News