← Back to Insights

Decision

12 min read

Copilot was 'for entertainment purposes only': is AI accuracy still a reason not to start?

By Ben Gould · Published 28 October 2026

AI Accuracy
Reliability
Governance
Human-in-the-Loop
AI Strategy

No. Accuracy is a reason to design the work differently, not a reason to wait. Every AI vendor's terms say the output may be wrong and that checking it is your job, and none of them will ever say anything else. The businesses getting value from it have stopped asking whether the model is right and started building the checking into the process.

Until this August, Microsoft's terms of use for its consumer Copilot contained a sentence that did the rounds in April: "Copilot is for entertainment purposes only. It can make mistakes, and it may not work as intended." It sat under a heading reading "IMPORTANT DISCLOSURES & WARNINGS", followed by "Don't rely on Copilot for important advice. Use Copilot at your own risk" (XDA Developers). Microsoft told the press it was "legacy language" from when Copilot launched as a search companion inside Bing, no longer "reflective of how Copilot is used today", and that it "will be altered with our next update" (TechCrunch).

It was. Microsoft said in April that the clause was going; the current terms are dated 18 August 2026 and the word "entertainment" is gone. The edit itself drew none of the attention the original did, which is a pity, because the replacement wording is the more instructive document. It now reads: "Copilot can make mistakes and it may not work as intended. Do not use Copilot as a substitute for professional advice. Always verify the accuracy of information presented by Copilot before you rely on it," and, in capitals, "WE DO NOT MAKE ANY WARRANTY OR REPRESENTATION OF ANY KIND ABOUT COPILOT" (Microsoft). Microsoft removed the joke and kept the position. That position, not the phrasing, is what a cautious buyer should be reading.

Does any AI vendor warrant accuracy?

None of them, at any price, on any tier.

Vendor and productWhat the terms say about accuracy
Microsoft Copilot, consumer"Always verify the accuracy of information presented by Copilot before you rely on it." No warranty "of any kind".
Microsoft 365 Copilot, businessOutput is Customer Data; customer responsible for use. Contract silent on accuracy. Docs: "aren't guaranteed to be 100% factual".
OpenAI, consumer and business terms"Output may not always be accurate." "You should not rely on Output from our Services as a sole source of truth or factual information." Evaluate it "including using human review as appropriate".
Anthropic, consumer terms"Outputs may not always be accurate and may contain material inaccuracies even if they appear accurate." Don't rely on them "without independently confirming their accuracy".
Anthropic, commercial termsCustomer "must notify its Users" that factual assertions "should not be relied upon without independently checking their accuracy".
Google, Gemini and Cloud AIMay "provide inaccurate or offensive content"; generative services "are not designed for or intended to meet Customer's regulatory, legal, or other obligations".

Sources: Microsoft, OpenAI, Anthropic consumer and commercial terms, Google.

Read as a set, the pattern is obvious. Microsoft's "entertainment" line was the outlier wording, not the outlier position. Anthropic's commercial terms go a step further than the rest and push the duty down the chain: a business using Claude has to tell its own staff not to trust factual claims without checking. If you've been waiting for a vendor to stand behind the output before you build on it, you're waiting for a contract that will never be written. The reason it won't is worth understanding, because it tells you what to do instead.

Is hallucination still a problem, or was that 2024?

Both, depending on what you ask the model to do, and the difference is the whole answer.

The improvement is real. OpenAI's GPT-5 system card reports that its standard model has "a hallucination rate 26% smaller than GPT-4o" at the level of individual claims, and the reasoning version "65% smaller than OpenAI o3" (OpenAI, GPT-5 system card). Every lab publishes something similar every few months, and the trend is in the right direction.

The same system card, a few lines down, shows the limit. On a benchmark of hard factual questions answered from memory, without web access, the models still got between 40% and 47% of their attempted answers wrong. When the model is asked to produce facts it has no source for, it still produces them confidently and still gets a great many of them wrong. OpenAI's own researchers explained why in a paper last year: models hallucinate because "the training and evaluation procedures reward guessing over acknowledging uncertainty" (Kalai et al.). The model is scored like a student who loses nothing for a wrong answer and so never leaves a question blank. That is a property of how these systems are built and measured, not a bug awaiting to get fixed.

Now look at the other kind of task. Vectara's hallucination leaderboard, updated last week, measures something different: how often a model invents material when summarising a document it has been given. The best frontier models fabricate in 6.5% to 12% of summaries, and some small, cheap models do better than that at around 3% (Vectara). That is still not zero, and 6% of summaries containing an invention is a real number if you are sending them to clients. But it is a different order of problem from 40%, and the gap between the two figures is the lesson. Give the model the source and the error rate falls by a factor of five or more. Ask it to remember, and it guesses.

Why does the gap in the terms matter more than the hallucination rate?

Because the rate is improving and the gap is not, and the gap is where responsibility lands.

Three things happened in the last eighteen months that make this concrete. In June 2025, the High Court dealt with two cases in which lawyers had cited authorities that did not exist, five in one case and eighteen in the other, and warned that freely available tools "may cite sources that do not exist" and are "not capable of conducting reliable legal research" (Ayinde v Haringey). In October 2025, Deloitte refunded the final instalment of a A$440,000 report to the Australian government after it was found to contain fabricated references and a made-up judicial quotation; the report's own appendix disclosed the use of a GPT-4o toolchain (City AM, TechSpot). And in August this year, the day before Microsoft's new terms took effect, the Solicitors Regulation Authority published a warning notice saying that AI tools "can produce 'hallucinations', generating fictitious cases, references or seemingly factual assertions", that solicitors "remain personally responsible for their accuracy, regardless of the tools used in drafting", and that "reliance on an output of AI would not be a suitable defence" (SRA).

Put the vendor terms next to the regulator's notice and they say the same thing from opposite ends. The vendor will not take responsibility for the output. The regulator won't let you pass it on. Whatever the hallucination rate does next year, that arrangement is permanent, and it is the actual constraint on how a business should use these tools. Not "can it be wrong", which it can, but "who carries it when it is", which is you. I made the same point about who owns what the AI produces in the context of an acceptable-use policy: if you send it, you own it. The T&Cs are just the vendors agreeing.

So is accuracy still a reason not to start?

Most UK businesses have already answered that by starting. The useful question is what they started on.

The British Chambers of Commerce found 54% of SMEs using AI in 2026, up from 35% a year earlier (BCC). The government's own adoption research found that concerns about output accuracy were common, but for most businesses "did not prevent or delay them from deploying AI"; the barriers that actually stop firms are not seeing a use for it and not having the skills (DSIT). And in Ipsos's polling this summer, 74% of Britons said they "do not trust AI outputs enough to use them without checking the work" (Ipsos).

That last number is usually quoted as a problem. I read it as the correct posture, held by three-quarters of the population without anyone having to train them. The error the cautious buyer makes is not distrust. It is treating distrust as a reason to do nothing, when it is a design requirement. If you don't trust the output, good, build the process so that trust is never required. That's a solvable engineering problem, and it is the same one every other unreliable component in a business has had solved for it: the new starter whose work gets reviewed, the supplier whose invoices get matched, the spreadsheet whose totals get reconciled.

What does the engineering answer look like?

Four things, in roughly this order, and none of them is "pick a better model".

  1. Give it the source. The five-fold difference between the closed-book and grounded figures above is the single biggest lever you have. An automation that reads your documents, your CRM records and your policies, and is told to answer only from them, is a different tool from a chatbot asked to remember. Retrieval has been shown to reduce fabrication since before the current generation of models (Shuster et al.), and the vendors' own guidance now says the same: restrict the model to the material provided and make it quote its evidence (Anthropic).
  2. Constrain the shape, then check the content. Every major API can now force the model's output into a fixed structure, a form with named fields rather than free prose (OpenAI). That guarantees the shape, not the truth. But a fixed shape is what lets ordinary software check the truth: does this invoice total match the line items, does this date fall inside the contract, does this case citation exist. Deterministic checks first, as I described in how to check what an AI agent actually did; a model's judgement only where a rule can't reach.
  3. Make "I don't know" an acceptable answer. The OpenAI paper's diagnosis points at the cure: the model guesses because guessing is never penalised. In your own system you can penalise it. Tell it to return "not found" rather than a best guess, route those cases to a person, and measure how often it happens. An automation that says "unsure" 8% of the time and is right the other 92% is far more valuable than one that is right 95% of the time and never tells you which 5%.
  4. Put the person where the risk is, not everywhere. The UK government's own AI playbook calls for "meaningful human control at the right stages" and says "humans validate any high-risk decisions influenced by AI" (GOV.UK). The important words are "at the right stages". A person reading every output is not a control, it's the old process with an extra step. A person approving everything irreversible, and sampling the rest, is. I've set out the three places a human belongs in a running automation; which one you need depends on what a wrong answer costs.

Then do the thing almost nobody does: measure the error rate on your own work. The Information Commissioner's position is that the "appropriate level of statistical accuracy" of a generative system "is linked to the specific purpose" it's used for (ICO). That is a polite way of saying there is no universal acceptable rate. A 3% error rate in a first-draft marketing summary is fine. A 3% error rate in VAT codes is a letter from HMRC. Decide what rate each use can tolerate, run a sample of real work through the system before it goes live, and compare. If you can't state the number, you haven't finished designing it.

Where does that leave a cautious buyer?

With a short list of uses that are safe now, a longer list that are safe with controls, and a few that aren't yet.

Drafting, summarising and research with a person reading the result before anyone else does: safe now, and the T&Cs are close to irrelevant, because the person is the control and always was. Extraction, classification and matching, where the output can be validated against a rule or a source system: safe with the controls above, and this is where most of the money is. Anything that makes a factual claim to a customer or a regulator with no person and no check between the model and the recipient: not yet, and perhaps not ever at a small firm, because the exposure is yours and the vendor has told you so in writing. The question was never whether to trust the model. It's whether you've built something that doesn't need to.

The bottom line

Microsoft deleted "for entertainment purposes only" in August and replaced it with "always verify the accuracy of information presented by Copilot before you rely on it". That edit is the whole story. No vendor warrants the output, no regulator will accept "the AI said so", and neither of those facts is going to change as the models improve, because hallucination is a property of how they are trained and scored. Accuracy is therefore not a reason to wait. It's the specification: ground the model in your own sources, force a structure you can check, let it say "I don't know", put a person at the points of real risk, and measure the error rate on your own work before you rely on it. The businesses doing that are getting value today. The ones waiting for a warranty will be waiting a long time.

If you'd like a second opinion on which of your processes can tolerate what, and how to build the checking in rather than bolt it on, that is what an AI consulting engagement starts with. A thirty-minute discovery call will tell you whether the thing you're cautious about is a real risk or a solved one.

Related reading

Or see how I put this into practice: services, case studies.