How to Check Sources in an AI-Assisted Report Before Send (Plus Disclosure Lines)

In July 2025, Australia's employment department published an independent review of its Targeted Compliance Framework prepared by Deloitte under an AU$440,000 (~US$290,000) contract (The Register, Oct 6, 2025). Dr. Christopher Rudge at the University of Sydney checked the footnotes and found over a dozen nonexistent papers plus a fabricated quote attributed to the federal court judgment Amato v Commonwealth (CTV News / AP, Oct 6, 2025). In October 2025, a revised edition quietly removed the fake cites and added a note disclosing Azure OpenAI GPT-4o, which Rudge called a "confession." Deloitte agreed to repay its final contract instalment (exact dollar figure undisclosed). In March 2026, Chartered Professional Accountants Newfoundland and Labrador opened an investigation into Deloitte Canada over fake citations in a 526-page, C$1.6 million health workforce plan (The Independent NL, Mar 6, 2026).

Phantom footnotes are not rare edge cases. On June 22, 2023, Judge P. Kevin Castel imposed a $5,000 Rule 11 sanction in Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), after counsel used ChatGPT to cite six fake aviation cases and asked ChatGPT if those cases were real (CourtListener). As of October 29, 2026, Damien Charlotin's database tracks 2,241 tribunal decisions worldwide involving hallucinated citations (1,514 US; 1,292 pro se litigants; 886 lawyers) (Charlotin Database; MSBA). Penalties now top six figures: in March 2026, the Sixth Circuit ordered $116,315.09 in combined sanctions (16,315.09 fees plus a100,000 fine) over unverified AI filings (JD Supra, Mar 24, 2026).

Desk workers outside law face the same risk. In the KPMG and University of Melbourne global study (n > 48,000 across 47 countries, Nov 2024 to Jan 2025), 66% of the respondents relied on AI output without checking accuracy, and 56% of the workers reported making work mistakes due to AI (KPMG Study, Apr 2025). Source audits sit on the practical side of the two meanings of AI safety at work. Below is why smooth AI text fools smart readers, the error rates of AI search and RAG tools, a five-check audit with time math, and four disclosure lines.

What 1920s magazine fact-checkers knew that AI report writers keep relearning

Magazine editors separated smooth prose from verified truth a century before chatbots existed. In the 1920s and 1930s, Time and The New Yorker built research desks where checkers underlined every name, date, quote, and number in colored pencil. Their rule was blunt: clean grammar proves zero factual accuracy. For decades, offices cheated on that rule because human haste left clues. When a tired analyst guessed a figure at 1:00 AM, the surrounding sentences looked choppy. Rough prose warned managers to check the math.

Autoregressive language models broke that warning signal. Next-token prediction optimizes fluency whether or not a real record exists. Think of an unverified AI draft like fresh drywall hung and painted over concrete footings nobody inspected: smooth paint hides missing rebar just like solid steel. That analogy holds for visual skimming, though it breaks in one helpful way. Tearing open drywall ruins the wall, whereas checking digital footnotes leaves clean paragraphs intact when primary files back them up.

OpenAI's research shows why models guess instead of staying silent. In a September 5, 2025 study on hallucinations, OpenAI researchers explained that binary pretraining and benchmarks award zero points for "I do not know," punishing honest abstention just like wrong guesses (OpenAI Research, September 5, 2025). On SimpleQA, gpt-5-thinking-mini abstained on 52% of the queries and erred on 26% of the test, while an earlier model (o4-mini) abstained on 1% of the queries and got 75% of the benchmark wrong. Prompting a bot to "add three sources per section" pushes it into low-abstention guessing. Compare task error rates in our guide on how often AI is wrong by task.

Why AI search engines and RAG tools do not save you from citation errors

Connecting a model to web search or retrieval-augmented generation (RAG) cuts pure invention, yet audits from 2024 to 2026 prove search-backed AI still misattributes, ungrounds, and inflates sources. Blue footnote icons inside an AI chat window do not equal verified evidence.

Four benchmarks measure how often search and RAG systems fail on citations:

1. AI news search (Columbia Tow Center, March 2025): Jazwinska and Chandrasekar tested eight live AI search engines on 200 news quotes across 20 outlets (n = 1,600 queries, tested February 2025). The tools gave wrong citation answers on over 60% of the queries, from a 37% error rate on Perplexity to a 94% error rate on Grok 3, where 154 out of 200 links broke (CJR Tow Center, March 6, 2025). Paid tiers were more confidently wrong than free tiers behind paywalls.
2. Ungrounded AI Overviews (Oumi / NYT, April 2026): Oumi tested n = 4,326 SimpleQA searches on Gemini 3-era Google AI Overviews for The New York Times. Answers were right in 91% of the searches, yet 56% of the accurate answers were ungrounded, meaning linked pages did not back the claim (TechRepublic, April 21, 2026).
3. Commercial legal RAG (Stanford RegLab & HAI, May 2024): Across n > 200 pre-registered legal queries, Magesh et al. tested legal RAG tools marketed as "hallucination-free." Lexis+ AI and Ask Practical Law AI hallucinated on over 17% of the queries each, while Westlaw AI-Assisted Research erred on over 33% of the queries (over 34% in HAI's summary) (Stanford HAI, May 23, 2024; arXiv:2405.20362). Note the May 2024 vintage: vendors shipped updates since, though no independent peer-reviewed replication of matching scale has appeared in 2025 or 2026.
4. Scientific overgeneralization (Peters & Chin-Yee, April 2025): Even when given the real text, bots inflate claims. Auditing n = 4,900 summaries of scientific papers across 10 LLMs (arXiv:2504.00025v1), Peters and Chin-Yee found models stripped hedges and overgeneralized findings in 26% to 73% of the summaries, nearly five times the human rate (odds ratio 4.85) (arXiv:2504.00025v1). Prompts asking for "accurate" summaries doubled overgeneralizations compared with plain prompts.

Test any internal RAG tool on your own files first with our 50-example spreadsheet method for testing an AI tool.

The ed.cafe Five-Mode Citation Taxonomy: five ways AI footnotes break in the wild

Under the ed.cafe Five-Mode Citation Taxonomy from No Worker Left Behind, AI footnotes break in five ways. Only Mode 1 looks broken at a glance; Modes 2 through 5 hide behind live URLs and real author names.

Mode 1: Phantom URLs, dead DOIs, and composite ghost titles

Mode 1 splices a real scholar's name, a real journal, and a plausible year into a paper that never existed. Two May 2025 stories showed how easily Mode 1 clears editorial desks. On May 18, 2025, a syndicated summer insert in the Chicago Sun-Times and Philadelphia Inquirer recommended 15 books; 10 out of 15 titles were fake, generated by AI for a freelancer (NPR, May 20, 2025). Days later, reporters checking the White House MAHA report found at least seven cited studies among 500+ footnotes that did not exist (CBS News, May 30, 2025).

Mode 2: Real URL, ungrounded or unrelated claim

Mode 2 opens a real web page that says nothing about your claim. In the April 2026 Oumi audit of 4,326 Google AI Overviews, 56% of the right answers linked to pages that never backed up the sentence (Oumi / NYT, April 2026). Link checkers that only test for HTTP 200 miss 100% of the Mode 2 errors.

Mode 3: Real document, fabricated verbatim quotation

Mode 3 cites a real ruling, statute, or report while putting invented words inside quotation marks. Deloitte Australia's July 2025 report hit Mode 3 when it cited the real federal case Amato v Commonwealth with a fabricated judicial quote (The Register, October 2025). Every quoted phrase in an AI draft requires a Ctrl+F (Cmd+F) search inside the primary file.

Mode 4: Real source, inverted denominator or scope inflation

Mode 4 turns a narrow pilot number or hedged correlation from a real paper into an unqualified rule. Peters and Chin-Yee documented this across 4,900 summaries, where LLMs dropped limits like "in this subgroup" in 26% to 73% of the trials (Peters & Chin-Yee, April 2025). Check that percentages keep their real n, population, and year.

Mode 5: Real rule, superseded statute or expired guidance

Mode 5 cites an authentic policy page that was later amended, delayed, or repealed. Early summaries of the EU AI Act stated all high-risk workplace rules take effect in August 2026, missing Regulation (EU) 2026/1744 (published July 24, 2026), which moved Annex III high-risk enforcement to December 2, 2027 while keeping Article 50 transparency live on August 2, 2026 (EUR-Lex 2026/1744). Municipal bots fail here too: New York City spent ~$500,000 on its MyCity business chatbot before shutting it down in February 2026 after it told employers they could keep worker tips and refuse housing vouchers (The Markup, January 30, 2026).

Adult teams can practice spotting all five modes together in our 75-minute first hallucination lab for adults.

Worked time-budget math: why source-first drafting beats retro-checking AI footnotes

Analysts skip source checks when managers budget zero minutes for verification. Minute-by-minute math on a 12-page, 24-citation report or grant narrative shows why telling AI to "draft the report with sources" takes 45 minutes longer than gathering real sources first:

• Workflow A ("Draft-First, Verify-After"): You prompt AI to write a 12-page draft with 24 citations (15 minutes). Because 35% to 60% of the AI-chosen footnotes carry Mode 1 to Mode 5 flaws, checking 24 unfamiliar links averages 6.5 minutes per citation (156 minutes): hunting paywalls, finding 9 ungrounded or dead links, pulling replacement papers, and rewriting broken claims. Total: 171 minutes (2 hours, 51 minutes).
• Workflow B ("Source-First, Draft-Second"): You gather 24 primary PDFs or statutes first, logging each URL, page, and n (72 minutes, or 3.0 minutes per source). You feed only verified excerpts to AI to tighten prose (24 minutes), then run the five-check audit against open PDFs at 1.25 minutes per citation (30 minutes). Total: 126 minutes (2 hours, 6 minutes).

Skipping verification shifts that cleanup cost onto peers and clients. In September 2025, BetterUp Labs and the Stanford Social Media Lab surveyed US desk workers on "workslop," polished AI output lacking substance that forces coworkers to repair it (BetterUp Labs, Sept 2025). Note two caveats: vendor BetterUp funded the survey, and releases list two slightly different totals (n = 1,004 in text vs n = 1,150 in the graphic) plus cleanup averages of 1 hour 56 minutes vs 1 hour 51 minutes. Even with those caveats, 40% of the surveyed workers received workslop in the prior month, costing a survey-modeled $186 per employee per month. Pair this audit with our 12-point rubric for checking and fixing AI work samples.

Copy-and-paste checklist: The ed.cafe Five-Check Citation Audit & Disclosure Stamp

Every AI-assisted brief, client report, or grant narrative needs two gates before send: five checks across 100% of the citations and one plain sentence naming how AI helped. Copy or print The ed.cafe Five-Check Citation Audit & Disclosure Stamp from ed.cafe (No Worker Left Behind) below.

THE ED.CAFE FIVE-CHECK CITATION AUDIT & DISCLOSURE STAMP
Publisher: ed.cafe (No Worker Left Behind, Boston 501(c)(3))
Rule: Never ask an AI chatbot or AI search box to verify its own citations.
      Open every primary source yourself in an independent browser tab.

PART 1: FIVE CHECKS REQUIRED ON 100% OF CITES BEFORE SEND
Run Checks 1-5 on every footnote, URL, statute, case, and number:

[ ] CHECK 1 - EXISTENCE & INDEPENDENT RESOLUTION (Defeats Mode 1)
    - Open the exact DOI, court docket, government URL, or ISBN directly
      (via Crossref, PubMed, CourtListener, EUR-Lex, or agency archive).
    - Confirm exact author names, publication title, outlet, and year.
    - HTTP 200 alone is NOT enough; read past the title and paywall.

[ ] CHECK 2 - CLAIM-LEVEL GROUNDING ON EXACT PAGE (Defeats Mode 2)
    - Locate the specific page, section, or table inside the primary file.
    - Confirm the source explicitly proves the claim in your sentence,
      not merely the general topic or background vocabulary.

[ ] CHECK 3 - WORD-FOR-WORD QUOTATION MATCH (Defeats Mode 3)
    - Run Ctrl+F / Cmd+F inside the primary document for every phrase in
      quotation marks.
    - If a quote was reconstructed from notes or summary text, remove the
      quotation marks or replace with the verified verbatim string.

[ ] CHECK 4 - DENOMINATOR, SAMPLE & HEDGE INTEGRITY (Defeats Mode 4)
    - Check every percentage, dollar figure, multiplier, and odds ratio.
    - Keep the real denominator in prose (sample size n, year collected,
      geography, population, and whether vendor-sponsored or peer-reviewed).
    - Restore scientific hedges ("associated with" != "caused").

[ ] CHECK 5 - STATUTORY & VERSION CURRENCY (Defeats Mode 5)
    - Check whether the statute, policy, agency guidance, or software doc
      was amended, stayed, repealed, or superseded after publication.
    - Record the exact version number or "verified as of [Date]" note.

PART 2: SIGN-OFF LEDGER (PASTE INTO TICKET OR COVER SHEET)
Document Title: _________________________________________________
Total Citations / Data Claims Audited: _____ / _____ (Must equal 100%)
Citations Replaced or Rewritten During Audit: _____
AI Models / Tools Used in Workflow: _____________________________
Human Reviewer Signature & Date: ________________________________

PART 3: FOUR PLAIN-ENGLISH AI DISCLOSURE STAMPS (PICK ONE & EDIT)

1. INTERNAL MEMO OR BRIEFING NOTE:
   "Method note: Staff used [Tool/Model Name] to organize outline notes
    and tighten prose drafts; [Author Name] independently verified every
    source link, quotation, and numerical figure against primary records
    on [Date]."

2. EXTERNAL CLIENT DELIVERABLE OR CONSULTING REPORT:
   "Disclosure: Portions of the prose editing and table formatting in this
    report were assisted by [Enterprise Tool Name] under zero-training
    workspace controls. No confidential client records were submitted to
    public models, and [Lead Analyst Name] verified 100% of citations,
    quotations, and calculations against primary sources."

3. NONPROFIT OR RESEARCH GRANT APPLICATION:
   "AI Assistance Statement: [Organization Name] used [Tool Name] for
    structural outlining and readability editing only. Program design,
    budget math, community data, and all cited literature were authored
    and primary-source verified by [PI / Grant Writer Name] in compliance
    with funder guidelines."

4. PUBLIC REPORT, POLICY BRIEF, OR EXTERNAL PUBLICATION:
   "Editorial & AI Transparency Note: Generative AI ([Tool Name]) assisted
    with copy-editing and layout formatting during production. Human
    editors at [Organization Name] bear full editorial responsibility and
    verified every cited study, statute, quotation, and statistic against
    its original source prior to publication."

Why hiding AI use turns a fixable typo into a "confession" scandal

Notice what turned Deloitte Australia's October 2025 revision from a routine errata note into global news. Consultants have made footnote typos for decades. The scandal came from the sequence: an AU$440,000 report shipped with zero mention of AI, a scholar exposed fake papers and an invented court quote, and only then did an appendix note admit Azure OpenAI GPT-4o was used (The Register, October 2025). Retroactive disclosure reads like getting caught.

Staff hide AI help because workplaces demand instant drafts while shaming tool use. In the KPMG and University of Melbourne global study (n > 48,000), 57% of the workers using AI hid their AI use and presented AI output as their own (KPMG Global Study, April 2025). When half a team hides AI use, nobody asks a peer to check a shaky footnote. Break that secrecy habit with our guide on why teams hide AI use and a 30-day amnesty plan. Saving draft history and source tables also protects writers when noisy detectors misfire; see what to save if accused of using AI unfairly.

Procurement rules and statutes now pair human verification with explicit labelling:

• Massachusetts Policy AI.001 v1.1: Revised March 17, 2025, Massachusetts policy requires Executive Department staff and covered engagements to fact-check generative AI outputs against primary sources and label AI-generated content before distribution (Mass.gov Policy AI.001 PDF).
• EU AI Act Article 50(4): Live August 2, 2026, Article 50(4) requires deployers publishing AI-generated public-interest text to disclose artificial generation unless the text underwent human review and a natural or legal person holds editorial responsibility (EU AI Act Article 50). Stamp #4 above names who holds that accountability.

Inside the ed.cafe Staff Room and our circles at No Worker Left Behind (where Circle #1, a free 30-chair Boston cohort taking the Google AI Professional Certificate via the Massachusetts AI Hub, forms in October 2026 ahead of its December 1, 2026 close), learners build source-first tables on real workplace drafts.

Where this stops being true

Tab-by-tab checks on 100% of citations plus a disclosure line cover almost all human-read reports, with three exceptions:

• Explicit AI bans by courts, funders, or clients: A disclosure stamp never overrides a judicial standing order, grant peer-review rule, or NDA banning AI tools outright. Check RFP rules and contracts first.
• High-volume pipelines (>10,000 outputs/day): Manual Ctrl+F checks cannot scale to 50,000 daily summaries. High-volume pipelines require DOI/URL APIs (Crossref, OpenAlex, CourtListener), span matching, and statistical QA sampling.
• Offline archival records: When citing unindexed paper archives, oral histories, or restricted charts, browser checks cannot verify quotes. Attach scanned pages or a two-person sign-off sheet in the project folder.

Questions people ask

Key terms

• Citation hallucination (phantom authority): A fabricated AI reference blending plausible authors, outlets, dates, or docket numbers into a nonexistent work.
• Ungrounded citation: A real link or paper attached as a footnote whose text does not contain or prove the claim in the sentence.
• Overgeneralization bias: The habit of LLMs summarizing scientific studies to strip qualifiers and sample limits (arXiv:2504.00025v1).
• Source-first drafting: Logging verified primary files first before using AI strictly to organize outlines or edit prose from those excerpts.
• Abstention threshold: A model or workflow rule making an AI system state "I do not know" rather than guessing when evidence is absent.
• Editorial responsibility carve-out: EU AI Act Article 50(4)'s exemption from machine labels when human review occurs and a named person or entity holds editorial responsibility.

Sources

1. The Register, "Deloitte issues refund for error-ridden Australian government report that used AI" (Oct 6, 2025): The Register.
2. CTV News / AP, "Deloitte to partially refund Australian government for report with apparent AI-generated errors" (Oct 6, 2025): CTV News.
3. The Independent NL, "Accounting watchdog to investigate Deloitte over fake citations in $1.6-million healthcare report" (Mar 6, 2026): The Independent.
4. S.D.N.Y., Mata v. Avianca, Inc., 678 F. Supp. 3d 443, No. 1:22-cv-01461 (June 22, 2023): CourtListener.
5. Damien Charlotin, "AI Hallucination Cases Database" (2,241 decisions as of Oct 29, 2026): Charlotin Tracker.
6. Minnesota State Bar Association, "Confronting the Hard Numbers Behind Lawyers, AI, and Phantom Precedents" (2026): MSBA Analysis.
7. JD Supra, "6th Circuit Court of Appeals Issues Six-Figure Sanction Over Suspected AI Hallucinations" ($116,315.09, Mar 24, 2026): JD Supra.
8. CJR Tow Center, Jazwinska & Chandrasekar, "We Compared Eight AI Search Engines. They're All Bad at Citing News" (n = 1,600, Mar 6, 2025): CJR Study.
9. TechRepublic on NYT / Oumi Google AI Overviews audit (n = 4,326 searches, Apr 21, 2026): TechRepublic.
10. Stanford HAI & RegLab, Magesh et al., "AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries" (n > 200, May 23, 2024): Stanford HAI and arXiv:2405.20362.
11. Peters & Chin-Yee, "Generalization bias in large language model summarization of scientific research," Royal Society Open Science (n = 4,900, Apr 2025): arXiv:2504.00025v1.
12. OpenAI Research, Kalai et al., "Why Language Models Hallucinate" (Sept 5, 2025): OpenAI Index.
13. NPR, "Syndicated summer reading list in 'Chicago Sun-Times' recommends fake books generated by AI" (May 20, 2025): NPR.
14. CBS News, "RFK Jr.'s MAHA report cited studies that appear not to exist" (May 30, 2025): CBS News.
15. KPMG & University of Melbourne, Trust, Attitudes and Use of AI: Global Study 2025 (n > 48,000, Apr 2025): KPMG Release.
16. BetterUp Labs & Stanford Social Media Lab, "The Hidden Costs of Workslop" (vendor-sponsored, n = 1,004 text / n = 1,150 graphic, Sept 2025): BetterUp Blog.
17. Massachusetts EOTSS, Enterprise Use and Development of GenAI Policy, AI.001 v1.1 (Mar 17, 2025): Mass.gov PDF.
18. EU AI Act Article 50 & Regulation (EU) 2026/1744 (July 24, 2026): Article 50 and EUR-Lex.
19. The Markup, "Mamdani to Kill the NYC AI Chatbot We Caught Telling Businesses to Break the Law" (Jan 30, 2026): The Markup.