AI overestimates human rationality, study finds

AI systems may be making a basic error when dealing with humans. They assume people are far more rational and strategically sophisticated than they actually are, according to a new study by researchers from the Higher School of Economics and the University of Lausanne, recently published in the Journal of Economic Behavior & Organization.

The researchers tested several major large language models, including GPT-4o, GPT-4o Mini, Claude-Sonnet-4, Gemini-2.5 Flash and Llama-4 Maverick, using a classic economic experiment known as Keynes’s “beauty contest”, or the “guess the number” game. In the game, players choose a number between 0 and 100. The winner is the one whose number is closest to a fraction of the group’s average. Winning depends on predicting how others will think.

To compare AI and human behaviour, the team reproduced the conditions of 16 earlier experiments that had been conducted with real participants. The AI models were given the same game rules and detailed descriptions of their supposed opponents, including first-year economics students, conference attendees, individuals with strong analytical skills, and participants experiencing various emotional states.

The results showed that the models consistently expected their opponents to think in highly sophisticated ways. They adjusted their choices depending on who they believed they were facing, selecting numbers closer to theoretical predictions when matched with experts and higher numbers when facing less experienced groups. However, this tendency often worked against them because human players usually do not apply such deep strategic reasoning.

The study found that while the AI systems understood the game and demonstrated strategic thinking, they frequently “played too cleverly” and made poor predictions about real human behaviour. The models also failed to identify dominant strategies in simple two-player versions of the game.

The authors conclude that current AI systems may struggle in real-world decision-making tasks because they model human behaviour as more rational than it actually is.