Background & Context§
A new study from researchers at three European universities has quantified a troubling side effect of AI assistance: the erosion of critical thinking. Led by Valerio Capraro (University of Milano-Bicocca), Chiara Marcoccia (École Normale Supérieure), and Walter Quattrociocchi (Sapienza University of Rome), the experiment deliberately used an AI model that was usually wrong on a set of visual trivia questions. The goal was to isolate whether the mere presence of AI advice — even unreliable advice — alters human cognitive behavior. The results, published in a preprint, show that participants not only accepted erroneous AI answers but also became less willing to admit uncertainty and paradoxically more confident in their decisions.
The stakes are high. With AI increasingly embedded in education, search, and decision-making, the study suggests that reliance on AI may systematically suppress the human capacity to say "I don't know" — a cornerstone of intellectual humility and learning. This research adds to a growing body of evidence that AI tools can induce "cognitive surrender," a term coined by Wharton researchers earlier this year.
The News: What Happened Exactly§
The experimental design was elegant. The researchers selected a set of questions where large language models, particularly Google's Gemma 3.5 Flash (referred to as Step 3.5 Flash in the study), systematically fail: visual details from films, such as the color of a team's uniform in Bend It Like Beckham. By using a model that was usually wrong on these queries, any reduction in judgment could not be attributed to sensible delegation to a reliable tool. Participants were randomly assigned to either a control group with no AI access or a treatment group that could consult the AI before answering.
The results were stark. In the no-AI baseline, participants suspended judgment (answered "I don't know") 44% of the time and achieved 27% accuracy. With AI advice, the willingness to say "I don't know" collapsed to just 3%, and accuracy fell to 9% — a 66% relative drop. Meanwhile, confidence soared from 30% to 76%. As Capraro summarized: "People became much worse, the accuracy was only one third, but they were twice as confident." Some participants who would have answered correctly on their own consulted the AI and became wrong.
Even monetary incentives — designed to motivate careful reasoning — only partially restored critical judgment. With financial stakes, willingness to admit ignorance rose from 3% to 8%, and accuracy from 9% to 16%, but both remained far below the non-AI baselines of 44% and 27%. This indicates that the cognitive effect is not easily overridden by extrinsic motivation. The study echoes findings from Wharton researchers who earlier this year found that people accepted incorrect AI answers 80% of the time while reporting higher confidence than those working without AI. The new study sharpens the point: the mere availability of AI suppresses the cognitive habit of recognizing one's own ignorance.
Capraro expressed particular concern about children, who are developing critical thinking skills while being exposed to AI systems that are designed to answer confidently, never to admit uncertainty. Google's AI search overhaul, which replaced traditional links with AI-generated summaries, exemplifies this design choice. Common Sense Media recently called that design an "unacceptable risk" for students. The pattern is consistent: AI products are optimized to produce answers, not to say "I don't know." The humans using them are learning to do the same.
Historical Parallels & Similar Incidents§
The phenomenon of technology suppressing human cognitive skills is not new, but AI intensifies it in a novel way. A historical parallel can be drawn to the widespread adoption of GPS navigation systems. Before GPS, drivers relied on maps, landmarks, and mental mapping — skills that required active reasoning and memory. Studies in the 2000s and 2010s showed that reliance on GPS reduced hippocampal activity and spatial memory. A 2020 study by Javadi et al. found that frequent GPS users had poorer route memory and were less likely to explore alternative routes. When the GPS gave a wrong instruction, many drivers trusted it implicitly, leading to navigational errors. However, the magnitude of the effect was smaller: even heavy GPS users retained some ability to self-correct when the device was obviously wrong (e.g., directing into a lake).
The current AI study reveals a more pernicious dynamic. Unlike GPS, which operates in a physical world with feedback (dead ends, traffic, lakes), AI advice in knowledge tasks often lacks immediate, unambiguous feedback. A wrong answer to a trivia question may never be detected by the user, so confidence can remain inflated. Moreover, the suppression of "I don't know" responses is uniquely human and foundational to learning. In GPS use, people still said "I'm lost" when they had no signal. With AI, the very act of acknowledging uncertainty is being extinguished.
Another parallel is the well-documented automation bias in decision-support systems. Research from the 1990s onward, particularly in aviation and medicine, showed that operators often over-rely on automated advice, even when it conflicts with their own knowledge. For example, a 1994 study by Parasuraman and Riley found that pilots sometimes followed erroneous autopilot commands. However, those systems were designed with fail-safes and training emphasized cross-checking. The new AI tools, by contrast, are mass-market products used without formal training, and their creators optimize for engagement and perceived intelligence, not for user metacognition.
The lesson from these parallels is that technology shapes cognition at a deep level. The GPS case eventually led to calls for "informed use" and some navigation apps added features to encourage landmark awareness. Similarly, the current findings argue for AI designs that incorporate uncertainty estimation, calibration nudges, or even occasional "I don't know" responses from the AI itself. Without such interventions, the erosion of critical thinking may accelerate, especially among younger users whose metacognitive skills are still developing.