Can You Trust AI Meeting Notes? Whisper Fabrications, Flipped Negations, and a 5-Minute Check

In June 2024, researchers led by Cornell computer scientist Allison Koenecke tested OpenAI's Whisper speech-to-text API across n = 13,140 vocal segments from TalkBank's AphasiaBank and control speakers (ACM FAccT 2024; arXiv:2402.08021). Roughly 1% of transcripts in the study contained entire fabricated phrases or full sentences that nobody in the room ever spoke. Worse, 38% of those fabricated segments in the study introduced explicit harms, including invented violent scenarios, false medical details, and fabricated racial commentary. Four months later, an October 2024 Associated Press investigation reported that a Whisper-based clinical scribe from Nabla had been used across more than 7 million patient visits while automatically erasing source audio right after transcription for privacy reasons (AP News, October 26, 2024). Once the audio vanished, clinicians had zero recording left to prove a dangerous chart line was invented out of thin air.

Most blogs ranking for transcription accuracy repeat unsourced vendor slogans (such as unverified "95% accuracy rate" banners or "low-60% summary rate" claims) without naming a single benchmark sample or explaining how whole sentences materialize out of room noise. Inside our workplace AI safety hub, this guide separates measured error rates from marketing for team leads, admins, HR investigators, clinicians, and educators. It provides operational quality-control guidance rather than legal advice. Below, you get the two-layer technical mechanism, peer-reviewed benchmarks, Federal Rules of Civil Procedure discovery traps, worked error math, and The ed.cafe 5-Minute Post-Meeting Summary Verification & Purge Routine.

Why AI meeting tools fail twice: speech-to-text fabrications stacked under summary inversions

Unchecked AI meeting notes fail in ways human note-takers never do because every automated recap runs through two separate generative models stacked end to end: an automated speech recognition (ASR) model turning sound into text, followed by a large language model (LLM) compressing text into bullets.

People picture automated transcription like a court reporter's stenotype machine: if someone mumbles through a cough, the tape might mishear "fifteen" as "fifty" or print [inaudible], yet never invent a paragraph nobody mentioned.

Where that stenotype analogy breaks down is generative decoding. Legacy engines matched acoustic phonemes against a dictionary; zero sound meant zero text. Modern speech models like OpenAI's Whisper pair an acoustic encoder with an autoregressive language decoder trained on web audio and subtitles (Koenecke et al., ACM FAccT 2024). During pauses, throat clearing, or HVAC hum, the acoustic signal drops and the decoder hallucinates plausible next words, from YouTube sign-offs ("Thanks for watching!") to invented clinical or violent lines. See the token math in why ChatGPT makes things up and the plain-English fix.

Then Layer 2 feeds 7,000 words of interrupting dialogue into an LLM prompt for "Decisions and Next Steps." Real speech carries sarcasm, pushback, and backtracked examples ("Wait, let's NOT cut the Q4 ad budget by a 20% rate until Friday"). Summarizers flatten hypotheticals into commitments, drop "not," or glue Speaker A's question onto Speaker B's tasks.

What peer-reviewed benchmarks measure across Whisper, clinical scribes, and meeting summaries

Vendor landing pages love quoting a single "95% word accuracy rate" benchmark measured on clean studio audiobooks. Real workplace calls look nothing like audiobooks, and sentence-level fabrication rates tell a very different story from Word Error Rate (WER).

Start with Word Error Rate itself. Suppose a 45-minute project meeting produces a sample of 6,000 spoken words, and an ASR engine achieves a 5% Word Error Rate (300 wrong words out of 6,000 words, or a 95% word accuracy rate across the call sample). A 95% word-level benchmark rate sounds like an A on an exam. Yet if just two of those 300 word errors turn "do not terminate" into "do terminate" and "$14,000" into "$40,000," the summary reverses the entire call. For task-by-task error baselines beyond audio, read how often is AI wrong: error rates by task.

Now inspect four primary empirical studies with explicit sample denominators:

1. Speech-to-text fabrications and disability disparity (n = 13,140 vocal segments): In Careless Whisper (ACM FAccT 2024), Koenecke, Choi, Mei, Schellmann, and Sloane tested 13,140 vocal snippets in Whisper. Roughly 1% of segments in the study contained full-phrase hallucinations absent from the audio, and 38% of those hallucinations in the study carried explicit harms (violence, false associations, or false authority). Speakers with aphasia, who speak with longer pauses between words, suffered significantly higher fabrication rates than control speakers. Using unreviewed ASR transcripts in HR interviews creates a direct fairness risk for workers with speech disabilities, stutters, or non-native cadence.
2. Meeting summary error taxonomy (n = 200 annotated summaries): In "What's Wrong? Refining Meeting Summaries with LLM Feedback" (arXiv:2407.11919), Kirstein, Ruas, and Gipp built the human-annotated QMSum Mistake benchmark (n = 200 meeting summaries: 169 erroneous summaries making up 85% of the benchmark sample and 31 error-free controls making up 15% of the sample) across nine error types, including hallucination, total omission, partial omission, and coreference errors. Multi-instance chain-of-thought verification boosted hallucination detection accuracy by up to 29.5% over single-instance benchmark baselines, cutting false-negative rates to 3.4% in their test runs.
3. Academic medical center clinical note audit (n = 356 notes): In a 2026 UC Davis Health study of n = 356 AI clinical notes (JMIR Medical Informatics, 2026), 11.5% of notes in the study contained hallucinations and 18% of notes omitted spoken facts.
4. Commercial ambient scribe audit (n = 565 notes): A 2026 NeurIPS audit across n = 565 commercial scribe records (NeurIPS 2026) found at least one verified failure in 31.3% of audited records. Clinicians can pair today's checklist with when an AI scribe wrote an exam you never did: a 5-minute note check.

The litigation discovery trap and the audio-deletion paradox

Leaving unreviewed AI transcripts and auto-summaries in cloud folders creates a quiet legal hazard: in a dispute, opposing counsel can subpoena every word the model guessed.

Under Federal Rule of Civil Procedure 34 (Fed. R. Civ. P. 34(a)(1)(A), Cornell LII), litigants may request designated electronically stored information (ESI), including raw Zoom/Teams transcripts, Slack AI recaps, and notetaker archives. Picture a vendor dispute 18 months later where an unedited AI summary emailed to six attendees states: "Agreed to waive milestone penalty if vendor ships by November." Even if your lead actually said "We would never agree to waive the milestone penalty," your organization must spend deposition hours impeaching its own archived file.

Yet erasing source audio instantaneously, as Nabla's clinical tool did across 7 million visits (AP News), creates the opposite trap, the Audio-Deletion Paradox:

• Delete audio at Minute 0, keep unreviewed transcript forever: Destroys ground-truth audio before any human checks the summary, locking ASR hallucinations permanently into your official archive.
• Keep both raw audio and raw transcript forever: Accumulates years of biometric voiceprints and candid cross-talk, multiplying breach and wiretap exposure (mapped in AI notetakers in meetings: policy, consent, and how to remove bots).
• The ed.cafe Two-Stage Retention Rule (7-to-14 Day Verification Window): Hold source audio and the raw machine transcript for 7 to 14 days restricted to the host solely to verify disputed numbers or lines during The ed.cafe 5-Minute Post-Meeting Summary Verification & Purge Routine. Once the human-verified 5-to-8 bullet summary is saved as the official record, auto-purge both raw audio and raw unedited transcript at Day 14 (unless under a legal hold).

Worked math: unchecked meeting notes across a 25-person organization

Suppose a 25-person firm auto-transcribes 30 meetings per week (1,500 meetings/year), averaging 40 vocal segments per call (60,000 segments/year):

• At a 0.5% segment fabrication rate across the call sample (half the ~1% rate in Koenecke et al.'s n = 13,140 Whisper study), 300 fabricated phrases per year enter raw transcripts.
• If 10% of the 1,500 LLM summaries (150 summaries/year in the company cohort, below the 11.5% hallucination rate in UC Davis's n = 356 note study) flip a negation or misassign an owner, the firm sends three corrupted decision records every week.
• A 5-minute host review on the 12 decision calls each week takes 60 minutes/week company-wide and blocks unreviewed errors.

Five high-risk meetings where AI transcription does not belong at all

Carve five room types out of AI transcription entirely:

1. HR disciplinary, termination, and investigation interviews: Witness pauses trigger higher Whisper silence fabrications (ACM FAccT 2024), and negation slips corrupt HR files. Use a second human HR note-taker.
2. Candidate interviews and promotion calibrations: AI interview summaries trigger four-year Automated Decision System archival rules in California and Illinois, disadvantage speech-disabled candidates, and risk violating Massachusetts M.G.L. c. 149, § 19B if add-ons score vocal stress.
3. Privileged legal strategy and incident bridges: Verbatim cloud transcripts risk privilege disputes and preserve speculative early theories unless counsel directs otherwise.
4. Board executive sessions and M&A talks: Corporate secretaries record formal votes and actions, not verbatim transcripts of open debate.
5. Unconsented clinical, 42 CFR Part 2, or IEP calls: Never run general office bots on behavioral-health or special-education calls outside statutory consent and BAA controls.

How to run a 5-minute post-meeting verification before hitting send

First, flip one admin switch in Zoom AI Companion, Teams, or Meet: disable automatic post-meeting email distribution to all invitees and send drafts to the host only. Within 30 minutes of call end, run five 60-second checks from The ed.cafe 5-Minute Post-Meeting Summary Verification & Purge Routine:

1. Minute 1 (Numbers and dates): Check every digit (15 vs 50, net 30 vs net 13, Q2 vs Q3) against the 10-second audio timestamp.
2. Minute 2 (Negations and conditionals): Scan Decisions for dropped qualifiers (not, never, unless, pending review) and demote brainstorms out of binding decisions.
3. Minute 3 (Owners and deadlines): Confirm every named owner verbally accepted the task aloud and delete guessed Friday deadlines.
4. Minute 4 (Quotes and silence gaps): Never paste a "verbatim" quote without replaying that exact timestamp.
5. Minute 5 (Header and 14-day purge): Send only the cleaned bullet summary with a verification stamp, and let your 14-day rule purge raw audio and raw transcripts.

Copy-ready template: The ed.cafe 5-Minute Post-Meeting Summary Verification & Purge Routine

Copy this routine into your team handbook, executive-assistant SOP, or project wiki.

THE ED.CAFE 5-MINUTE POST-MEETING SUMMARY VERIFICATION & PURGE ROUTINE
Admin Prerequisite: Set Zoom / Teams / Meet AI summaries to "Host Only" (NEVER
auto-email unreviewed drafts to all invitees or external guests).

PART 1: THE HOST'S 5-MINUTE CHECKLIST (RUN WITHIN 30 MINUTES OF CALL END)
[ ] MIN 1 (NUMBERS & UNITS): Checked every $, %, headcount, SKU, and date.
    Replayed 10-sec audio clip on any binding price, fee, or timeline figure.
[ ] MIN 2 (NEGATIONS & HYPOTHETICALS): Checked every "Decision" bullet for
    dropped words ("not", "unless", "pending", "if") and demoted brainstorms.
[ ] MIN 3 (OWNERS & DEADLINES): Confirmed every named owner verbally agreed
    aloud on the call. Deleted guessed Fridays/EOD dates never spoken.
[ ] MIN 4 (QUOTES & SILENCE GAPS): Replayed exact audio timestamp before using
    any direct quote (guards against ~1% Whisper pause fabrications).
[ ] MIN 5 (STAMP & SEND): Sent ONLY the cleaned bullet summary with header:
    "Summary drafted with [Tool] and verified by [Host] on [Date]. Reply within
    48 hours with any corrections. Raw audio/transcript auto-purges at Day 14."

PART 2: TWO-STAGE RETENTION & PURGE SCHEDULE (IT / ADMIN POLICY)
- Stage 1 (Days 0 to 14, Verification Hold): Keep source audio + raw transcript
  restricted to Host only to verify disputed numbers or lines.
- Stage 2 (Day 14, Automatic Purge): Auto-delete BOTH raw audio AND unedited
  transcript; keep ONLY the verified summary (suspend immediately on Legal Hold).

PART 3: 10-CALL SPOT-CHECK TALLY SHEET (AUDIT BEFORE BUYING)
Call # | Min | Wrong Numbers | Flipped Negations | Silence Fabrications | Speaker Swaps
1.     | ___ | ___ count     | ___ count         | ___ count            | ___ count
2.     | ___ | ___ count     | ___ count         | ___ count            | ___ count

Where this stops being true

Treating AI meeting notes as rough drafts with a 14-day raw-buffer purge fits general office, nonprofit, and client project calls, yet five boundary conditions override parts of this routine:

1. Active or anticipated litigation holds: Once a dispute, EEOC charge, demand letter, or investigation makes litigation reasonably foreseeable, suspend your 14-day auto-purge rule immediately for affected custodians under Fed. R. Civ. P. 37(e).
2. Public Open Meeting Laws: Municipal boards and state bodies governed by open-meeting laws (such as the Massachusetts Open Meeting Law, M.G.L. c. 30A, §§ 18-25) must follow statutory retention schedules rather than purging at Day 14.
3. Regulated financial calls: Broker-dealers and advisers under SEC Rule 17a-4, FINRA Rule 3110, or CFTC rules must preserve covered communications in immutable archives for multi-year windows.
4. Live accessibility captioning (ADA): Real-time captions used by deaf, hard-of-hearing, or neurodivergent staff are protected accommodations under the ADA (42 U.S.C. § 12112); keep live captions enabled without permanently storing unedited transcripts.
5. Informational scope: This guide provides operational quality-assurance steps grounded in published benchmarks, not legal advice on evidence preservation, wiretap law, or compliance.

Teams building verification habits can run a free group lab from the ed.cafe Staff Room or apply for Massachusetts Circle #1 (30 free chairs for the Google AI Professional Certificate through No Worker Left Behind, closing December 1, 2026) on our start page.

Questions people ask

Key terms

• Speech-to-text fabrication (ASR hallucination): When a generative speech model outputs fluent words or full sentences during silence, noise, or pauses that nobody in the room spoke.
• Word Error Rate (WER): A transcription metric dividing wrong words by spoken words, which can hide negation flips inside a 95% word accuracy rate across a call sample.
• Negation / conditional inversion: When an LLM drops qualifying words (not, unless, pending review) while compressing multi-speaker dialogue into bullets.
• Audio-deletion paradox: Erasing source audio immediately for privacy while retaining an unreviewed AI transcript that can no longer be checked against spoken audio.
• QMSum Mistake benchmark: A human-annotated dataset (n = 200 meeting summaries across 9 error types, Kirstein et al., arXiv:2407.11919) testing LLM error detection in recaps.
• Electronically Stored Information (ESI): Digital records, including cloud recordings, raw transcripts, and AI summaries, subject to civil discovery under Fed. R. Civ. P. 34.

Sources

1. Koenecke, Choi, Mei, Schellmann & Sloane, "Careless Whisper: Speech-to-Text Hallucination Harms," Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24) (n = 13,140 segments; ~1% fabrication rate; 38% harmful): ACM Digital Library and arXiv:2402.08021.
2. Burke & Schellmann, Associated Press, "Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said" (Nabla >7 million visits & audio deletion analysis, October 26, 2024): AP News.
3. Kirstein, Ruas & Gipp, "What's Wrong? Refining Meeting Summaries with LLM Feedback" (QMSum Mistake dataset, n = 200 summaries across 9 error types, 2024/2025): arXiv:2407.11919.
4. UC Davis Health / JMIR Medical Informatics, Clinical AI Encounter Note Evaluation (n = 356 notes; 11.5% hallucination rate, 18% omission rate, 2026): JMIR Med Inform.
5. NeurIPS 2026 Commercial Ambient Scribe Audit (n = 565 clinical records; 31.3% verified failure rate, 2026): NeurIPS 2026 Proceedings.
6. Cornell Legal Information Institute, Federal Rules of Civil Procedure Rule 34 (Producing Documents and Electronically Stored Information) & Rule 37(e) (Failure to Preserve ESI): FRCP Rule 34 and FRCP Rule 37.
7. Massachusetts General Laws, Open Meeting Law (M.G.L. c. 30A, §§ 18-25) & Lie Detector Statute (M.G.L. c. 149, § 19B): Mass.gov Open Meeting Law and MA Legislature § 19B.