AI can decode hidden links between human attitudes, even across unrelated topics

Prior studies have shown that AI-generated persuasive content can be difficult to detect, even when disclosures are provided. The current study suggests that such systems might also know what to say based on a nuanced understanding of user beliefs. The study also echoes broader concerns in the AI al

A new study from Arizona State University demonstrates that advanced language models like OpenAI’s GPT-4o are capable of predicting human attitudes across diverse and unrelated topics, even when no clear semantic similarity exists between beliefs. The findings challenge conventional assumptions about how artificial intelligence understands and mimics human social cognition and raise new questions about the implications for personalization, persuasion, and bias in AI systems.

The research, led by Ana Ma and Derek Powell of ASU’s School of Social and Behavioral Sciences, tested GPT-4o’s capacity to infer the relationships between individual human beliefs. Using a newly constructed dataset derived from Pew Research’s OPINIONQA benchmark, the authors found that GPT-4o could accurately replicate the correlation patterns between people’s views and predict individuals’ answers to attitude questions, even when those questions appeared unrelated on the surface.

The study "Can Large Language Models Predict Associations Among Human Attitudes?" tested whether GPT-4o could predict a person’s answer to one attitude question based solely on their responses to others. The 64 selected statements spanned topics ranging from climate change and gun control to personal relationships and philosophy of life. Importantly, the researchers deliberately examined whether the model could make accurate predictions when given input questions that bore little to no semantic resemblance to the target question.

In the first phase, 376 U.S. adults completed a comprehensive online survey where they rated their agreement with each of the 64 statements. Correlation analysis of the human data revealed that many attitudes were strongly associated, even when conceptually distant. For example, support for action on climate change positively correlated with opposition to gun proliferation, a pattern commonly observed in ideological clusters, but difficult to infer from language alone.

The same patterns were then tested against GPT-4o. The model was prompted with synthetic statements and asked to predict how a hypothetical respondent might answer a target question given their previous responses. Across more than 20,000 pairings, GPT-4o’s estimated correlations between beliefs closely matched the human results, with a Pearson correlation coefficient of 0.77 - a remarkably strong alignment.

While semantic similarity did influence the model’s performance to some extent, the study found that its predictive ability remained robust even when high-similarity items were excluded. When fed only dissimilar prompts (based on cosine similarity scores below 0.20), GPT-4o still achieved a strong correlation of 0.724 with actual human data, indicating a level of reasoning that went beyond superficial linguistic matching.

To validate the predictions, the researchers trained “oracle” models using random forests on the human dataset, establishing a theoretical upper limit of predictive accuracy. While GPT-4o generally performed better when prompted with semantically similar items, it still achieved high accuracy when given dissimilar items, outperforming even the oracle in some cases.

The study builds on previous work by Hwang et al. (2023) and Santurkar et al. (2023), which showed that LLMs could simulate human-like personas and replicate known demographic or ideological patterns. But unlike those studies, which often relied on semantically clustered data, Ma and Powell’s work introduces a harder test: can LLMs see the social logic in belief systems the way people do?

The answer appears to be yes, but with important caveats.

GPT-4o consistently overestimated the strength of correlations between beliefs, especially for pairs with weak or moderate actual associations. The authors attribute this to a bias toward binary extremes in the model’s output distribution, which showed a bimodal clustering around strong agreement or disagreement.

Furthermore, the researchers noted that semantic similarity still offered a modest but measurable performance boost, suggesting the model partially relies on language-based heuristics. However, its ability to outperform expectations in low-similarity scenarios supports the view that these models possess emerging capabilities in social inference.

This raises both opportunities and risks. On one hand, the ability to simulate human attitudes accurately could support the development of AI tools for education, public health messaging, or sociological research. On the other hand, it presents serious ethical challenges in areas such as advertising, political targeting, and manipulation. Prior studies have shown that AI-generated persuasive content can be difficult to detect, even when disclosures are provided. The current study suggests that such systems might also know what to say based on a nuanced understanding of user beliefs.

The study also echoes broader concerns in the AI alignment and safety community, particularly regarding the persuasive and manipulative capacities of frontier models. Persuasion often begins with an accurate model of a target’s beliefs, something GPT-4o now appears to approximate with surprising precision.