Cognitive Surrender: Why We Trust Chatbots Even When They’re Wrong

In early 2026, researchers at the Wharton School introduced the term “cognitive surrender” to describe accepting AI answers with little scrutiny. In their experiments, participants who consulted a chatbot followed its incorrect advice about 80% of the time.

The Wharton Study

Title: “Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender”.
Authors: Steven D. Shaw, postdoctoral researcher, and Gideon Nave, Associate Professor of Marketing, both at the Wharton School of the University of Pennsylvania.
Format: a Wharton School Research Paper published on SSRN.
Design: three preregistered experiments with 1,372 participants and 9,593 trials.
Task: an adapted Cognitive Reflection Test, a set of logic puzzles with intuitive but wrong answers.

Tri-System Theory

The paper extends the model popularised by psychologist Daniel Kahneman:

System 1: fast, intuitive thinking.
System 2: slow, deliberate reasoning.
System 3: artificial cognition outside the brain, provided by AI.

The authors distinguish two behaviours:

Cognitive offloading: handing off a specific task, such as a calculation, while keeping control of the reasoning.
Cognitive surrender: accepting AI output in a way that overrides both intuition and deliberation.

Study 1: The Manipulated Chatbot

440 participants solved seven trick logic puzzles, either alone or with a chatbot.
The chatbot was secretly programmed to give correct answers on some puzzles and confident wrong answers on others.
Using the chatbot was optional; participants consulted it on more than 50% of trials.
When they consulted it, they followed correct advice more than 90% of the time and incorrect advice about 80% of the time.

Accuracy in Study 1:

about 71% with correct AI advice;
about 46% when working alone;
about 31% with faulty AI advice.

Overall, correct AI raised accuracy by 25 percentage points, and faulty AI lowered it by 15 percentage points. Participants’ confidence increased with AI use, even after they followed its errors.

Studies 2 and 3: Time Pressure and Incentives

Study 2 (485 participants): a 30-second limit per puzzle lowered overall accuracy, but reliance on the AI remained strong.
Study 3 (450 participants): a 20-cent bonus per correct answer plus instant feedback raised the rejection rate of faulty advice from 20% to 42%. Most incorrect answers were still accepted.

Who Surrenders Most

Higher trust in AI and technology predicted more cognitive surrender.
Higher “need for cognition”, the tendency to enjoy effortful thinking, predicted more rejection of wrong outputs.
Higher fluid intelligence also predicted more rejection of wrong outputs.

The authors identify health care, law, education and management consulting as the highest-risk domains. They recommend interfaces that flag AI uncertainty and prompt users to verify answers before acting.

Related Research

Microsoft Research and Carnegie Mellon (CHI 2025): a survey of 319 knowledge workers, covering 936 examples of generative AI use at work, found that higher confidence in AI was associated with less critical thinking, while higher self-confidence was associated with more.
MIT Media Lab (2025): 54 participants wrote essays with an LLM, a search engine or no tools while EEG recorded brain activity. The LLM group showed the weakest brain connectivity and the lowest sense of ownership over their essays.
Stanford (Science, 2026): 11 large language models endorsed users about 49% more often than humans did. More than 2,400 participants rated the sycophantic AI as more trustworthy.

The Stanford findings show that your chatbot is a yes-man, a design tendency that makes cognitive surrender more likely.

Automation Bias Before Chatbots

Over-reliance on machines was documented long before generative AI:

Aviation (1999): In a study by Linda Skitka, Kathleen Mosier and Mark Burdick with 80 participants on a flight task, about 65% of those using an automated aid acted on deliberately false prompts.
Missed events: about 41% of aid users missed events the automation did not flag, compared with about 3% of participants without the aid.
Radiology (2023): In a study published in Radiology, 27 radiologists read mammograms with AI suggestions. When the AI was wrong, accuracy among inexperienced readers fell from almost 80% to under 20%, and among very experienced readers from 82% to 45.5%.

Research Organisations Online

Studies on AI and reasoning are published and discussed through university, nonprofit and research-institute websites. The .org extension, managed by Public Interest Registry since 2003, is the standard domain for such organisations; comparing the .org domain price is a typical step when launching a research initiative or digital-literacy nonprofit.