Humans overestimate AI’s ability to mimic their decisions

The study focuses on economic decision-making across four key domains: risk, time preference, social preference, and strategic interactions. Participants were first asked to make decisions in various economic scenarios and then predict how GenAI, specifically OpenAI’s GPT-4o, would make the same decisions when acting on behalf of a human user.

As generative artificial intelligence (GenAI) systems become increasingly integrated into economic decision-making, their alignment with human choices has become a crucial area of study. A recent paper titled "Human Misperception of Generative-AI Alignment: A Laboratory Experiment" by Kevin He, Ran Shorrer, and Mengjia Xia, published in 2025, investigates how accurately people perceive GenAI's decision-making alignment with human preferences. The study, conducted through an incentivized laboratory experiment, reveals a significant overestimation of GenAI's ability to mimic human choices, raising important implications for AI adoption, delegation, and trust in AI-assisted decision-making.

Overestimating AI alignment in economic decisions

The study focuses on economic decision-making across four key domains: risk, time preference, social preference, and strategic interactions. Participants were first asked to make decisions in various economic scenarios and then predict how GenAI, specifically OpenAI's GPT-4o, would make the same decisions when acting on behalf of a human user. The results demonstrated a consistent overestimation of GenAI alignment. On average, participants' predictions about AI decisions were much closer to their own choices than to the actual choices made by GenAI. This pattern suggests that people intuitively assume AI will make decisions similar to humans, a cognitive bias known as anthropomorphic projection.

At an individual level, the study found strong correlations between a person's own choices and their predictions of how AI would act in similar scenarios. This phenomenon, termed self-projection, indicates that individuals believe GenAI will align not just with general human behavior but specifically with their personal decision-making tendencies. However, GenAI's actual choices often diverged significantly from both individual and collective human expectations, highlighting a gap between perception and reality in AI behavior.

Implications of misaligned AI expectations

The implications of these misperceptions are far-reaching, particularly in areas where AI is increasingly used to assist or even replace human decision-making. If individuals incorrectly assume that AI will make choices that reflect their own preferences, they may delegate tasks inappropriately, leading to suboptimal outcomes. This over-reliance on GenAI could be particularly problematic in high-stakes domains such as financial investments, healthcare decisions, and legal judgments, where human values and contextual reasoning play a crucial role.

Furthermore, the study's findings suggest that improving AI alignment alone may not be sufficient to ensure effective human-AI collaboration. If users maintain inaccurate beliefs about AI behavior, even objectively better AI performance may lead to misinformed delegation decisions. Addressing this challenge requires improving AI transparency and user education to help people form more accurate mental models of how AI systems operate.

Theoretical and practical considerations

To understand the broader impact of these misperceptions, the researchers developed a theoretical model illustrating how anthropomorphic and self-projection biases influence delegation decisions. The model suggests that when individuals overestimate AI alignment, they are more likely to delegate decision-making to AI, even in situations where human judgment would be more appropriate. Conversely, when people's personal preferences are highly distinct from the average human decision, they may benefit from avoiding delegation to AI, as their misperceptions of alignment are less pronounced.

Practically, these findings highlight the need for AI developers and policymakers to address user misconceptions through clearer AI explainability features. Improving AI communication by explicitly outlining decision rationales and potential biases could mitigate the effects of these misperceptions. Additionally, AI literacy programs should be incorporated into educational and professional training to equip users with a more nuanced understanding of GenAI capabilities and limitations.

Future directions in AI alignment and Human-AI interaction

This study opens avenues for further research on how humans perceive and interact with AI decision-making systems. Future studies could explore interventions designed to reduce anthropomorphic and self-projection biases, such as interactive AI demonstrations that contrast human and AI choices in real-time. Moreover, understanding how experience with AI influences these biases can inform strategies for improving AI adoption and trust.

As AI continues to play a larger role in economic and strategic decision-making, ensuring that users have accurate expectations about AI behavior is essential. This research underscores the importance of bridging the gap between AI alignment efforts and human perceptions, paving the way for more informed and effective human-AI collaboration.