reminder and planning nudges barely move course completion at scale. The largest test, about a quarter of a million learners across 247 online courses, found that self-regulation prompts raised engagement in the first few weeks but not final completion. The best result in a second trial was 2.6 percentage points. So a program supporting adults through the free Google AI Professional Certificate should spend its support budget on a person who notices when a learner goes quiet. This article gives the evidence, then a one-message protocol that starts after seven days of silence.
The number that changes the question
The number is "an order of magnitude smaller."
That is how René Kizilcec of Cornell described what happened when well-known behavioral nudges were scaled up. His team tested them on roughly 250,000 students in 247 courses from Harvard, MIT and Stanford over two and a half years. Earlier, smaller studies had reported medium-to-large effects. At scale, the Cornell Chronicle reported in June 2020, "the intervention effects were an order of magnitude smaller." Kizilcec's summary: "Behavioral interventions are not a silver bullet."
That matters now. Massachusetts gives residents free access to Google's AI certificate until 31 December 2027, per the Massachusetts AI Hub. Career centers are deciding how to support thousands of self-paced learners. Automated reminders look cheap and scalable. The evidence says they buy very little completion.
The evidence ladder, weakest to strongest
Each rung below names the intervention, the population and the measured effect on completion.
RUNG INTERVENTION POPULATION EFFECT ON COMPLETION
1 Planning tool alone MOOC learners, Bangladesh (BIT) No significant effect
2 Self-regulation prompts: make a ~250,000 learners, 247 courses Early engagement up for a few
plan, reflect on barriers, pick (Kizilcec et al., PNAS 2020) weeks; no rise in final
someone to report to completion
3 Planning tool + weekly SMS with MOOC learners, Bangladesh (BIT) +2.6 points on a 35.5% baseline
progress feedback
4 Paying for a verified certificate edX/HarvardX/MITx, 2017-18 46% of verified learners vs
3.13% of all participants
5 Structure with stakes: certificate South Texas College (JFF guide) "at least 85%" in one semester
counted for half a course gradeRungs 1 and 3 come from the same trial. The Behavioural Insights Team ran it with Bangladesh's a2i programme on the MuktoPaath platform. BIT's report, published 20 August 2020, found the planning tool alone had no significant effect. Adding weekly SMS messages with progress feedback raised completion by 2.6 percentage points from a control baseline of 35.5%.
BIT also reports a 15-point effect "when looking only at learners who completed the planning tool." Treat that figure with care. Learners who finish a planning tool are already more motivated. That is selection bias, not a treatment effect.
Rung 2 is the big one. Its prompts included "social accountability, where they pick someone to hold them accountable for their progress," per the Cornell summary. The planning-type prompts boosted engagement early, "but the impact dwindled as the course progressed." The study's own abstract, as listed by MIT's Teaching Systems Lab, says self-regulation interventions "raised student engagement in the first few weeks but not final completion rates."
Rungs 4 and 5 are associations, not experiments. In the edX data reported by Inside Higher Ed, paying learners finished far more often. They also chose to pay. The South Texas figure comes from the Jobs for the Future implementation guide and combines stakes, a schedule and staff contact.
Why a prompt to pick a partner is not a person who checks
Rung 2 tested a one-time prompt. Learners named an accountability partner inside the course and planned what to tell them. Nobody verified that the partner ever called.
A counselor check-in is a different intervention. The staff member sees the silence, reaches out first and offers a specific fix. The JFF guide recommends that programs "provide consistent touchpoints from instructional teams and student support staff." It describes the University of Texas Permian Basin "hiring full-time career coaches who have completed a Certificate." Motlow State Community College invites certificate completers "to serve as TAs."
Be clear about the limit here. No trial in this set isolates a staff check-in for adults on a free AI certificate. The case for a person rests on two points. Cheap prompts are proven weak at scale. And Massachusetts already assigns the job. DCS Information 14-623, issued 27 February 2026, tells career center staff to "provide regular check-ins, encouragement, and troubleshooting throughout the training." The license administrator, the SPOC, is told to "run Coursera reports to inform counselors of their jobseeker's progress."
The check-in is already mandated. The open question is how to make it good. Kizilcec's advice to instructors points the same way: "understand and address the specific challenges in their learning environment."
The 7-day check-in protocol
Copy this card into your case notes template.
7-DAY CHECK-IN CARD (version 1, untested in a cohort)
TRIGGER 7 days with no Coursera activity on the SPOC's progress report.
Check the report on the same weekday every week.
WHO SENDS One named person the learner has met. Not a system address.
CHANNEL The channel the learner chose at intake (text, email, call).
MESSAGE One question, under 40 words. Example:
"Hi [name], it's [your name] from [center]. I noticed the AI
course has been quiet for a week. What got in the way? I have
15 minutes on [day] at [time] if that helps."
SLOT Offer one specific 15-minute slot within 3 days.
DAY 14 No reply: send one second message with a different slot.
DAY 21 No reply: one closing note. "Your access runs to 31 Dec 2027.
Reply anytime and we'll pick up where you stopped."
Then stop. Record the outcome.
RECORD Date sent, reply (Y/N), restart within 7 days (Y/N),
stated reason in the learner's own words.What not to say:
- Do not open with the deadline or the funder. Pressure is not the barrier you are looking for.
- Do not send a progress percentage on its own. A low number reads as a verdict.
- Do not ask "Are you still interested?" It invites a polite exit.
- Do not send three reminders in a week. Volume is the tactic rung 1 already tested.
Why stress belongs in the script. In a fall 2023 Gallup-Lumina survey of 6,015 enrolled college students, 54% of those who considered stopping named emotional stress, the top reason, per Community College Daily. Those are degree students, not certificate learners. The pattern still argues for a question about what got in the way, not a reminder about what is due.
How many learners one person can cover
No source in this ladder reports a caseload limit for check-ins. Work it out from your own time.
Illustrative example: a counselor carries 30 learners on the certificate. Suppose 10 go quiet in a given week. At 5 minutes per message and 15 minutes per call for the half who reply, that week costs 50 minutes of messages and 75 minutes of calls. Round numbers, not measurements. Replace them with your own reply rate after two weeks.
Switch to a group when individual calls repeat. If three learners in one week are stuck on the same exercise, one 45-minute group lab beats three separate calls. The JFF guide notes that WhatsApp, Slack and Discord "can provide an additional space for learners to interact" with peers and instructors.
Competing explanations and limits
Context differences. The BIT trial ran in Bangladesh on a national platform. The Kizilcec trials ran on Harvard, MIT and Stanford MOOCs with a global audience. Neither population matches a Massachusetts adult on a free state license.
The "dwindling" may reflect who stays. Engagement effects that fade over a course can mean the nudge wore off. They can also mean the learners it helped early left anyway for reasons a prompt cannot reach, such as work hours or childcare.
The protocol is untested. ed.cafe has run zero cohorts. Every timing on the card is a starting estimate.
What would falsify this article. A trial on adult workforce learners showing automated reminders matching a staff check-in on completion, at lower cost, would overturn the budget advice here.
What ed.cafe will measure
When Circle #1 runs, ed.cafe will log every check-in sent under this card: the reply rate, the share who restart within 7 days and the reasons learners give. Results will be published alongside the cohort's completion count, including the parts that fail.
If you run this card at your center, bring your reply rates to the free monthly ed.cafe Staff Room once sessions begin.
Sources
- Study: No single solution helps all students complete MOOCs, Cornell Chronicle, 15 June 2020
- The limits of scalable interventions (Kizilcec et al., PNAS 2020 summary), MIT Teaching Systems Lab, accessed 15 October 2026
- Nudging online learners to complete their courses in Bangladesh, Behavioural Insights Team, 20 August 2020
- Study offers data to show MOOCs didn't achieve their goals, Inside Higher Ed, 16 January 2019
- Google Career Certificates Implementation Field Guide, Jobs for the Future, 31 August 2023
- DCS Information 14-623: Google AI & Digital Skills Partnership, MassHire Department of Career Services, 27 February 2026
- Emotional stress top reason students may withdraw, Community College Daily, February 2024
- Google certificates for Massachusetts residents, Massachusetts AI Hub (MassTech), accessed 15 October 2026
