AI supercharging phishing, vishing and deepfake fraud tactics

The authors note that social engineering has always exploited human psychology more than technical vulnerabilities. AI now strengthens this advantage by producing tailored content at scale, generating personalized messages within seconds and imitating human communication patterns with precision.

Artificial intelligence is reshaping the mechanics of cybercrime, lowering the skill barrier for attackers and expanding the reach of social engineering tactics, according to a new study published in Future Internet. Their findings reveal that AI-powered tools now allow even inexperienced actors to mount sophisticated digital fraud campaigns that mimic real-world attacks once reserved for highly skilled threat groups.

The peer-reviewed paper, titled “Social Engineering with AI”, presents a detailed examination of how modern AI systems strengthen three major forms of social engineering: spear phishing, vishing and malicious chatbot impersonation. The research is grounded in a real CFO fraud case in which deepfake audio and video were used to authorize a multimillion-dollar transfer, demonstrating how AI capabilities have begun to blur the line between human and machine deception.

The authors designed and executed three controlled experiments that replicate the strategic phases of the Hong Kong CFO attack while using only openly available AI tools. Their results show a clear trend: AI now acts as an amplifier that makes targeted manipulation faster, cheaper and more convincing. The report also raises significant questions for cybersecurity leaders, regulators and enterprises, who must now prepare for attack methods that evolve as quickly as the models powering them.

How AI is reshaping social engineering tactics

The research explores how the integration of AI changes both the method and the effectiveness of social engineering compared to traditional approaches. The authors note that social engineering has always exploited human psychology more than technical vulnerabilities. AI now strengthens this advantage by producing tailored content at scale, generating personalized messages within seconds and imitating human communication patterns with precision.

In spear phishing, attackers traditionally conducted manual reconnaissance to gather details about their targets. AI transforms this process by instantly searching online sources, compiling biographies and synthesizing personal background information into clean, structured narratives. The researchers tested several leading AI models with identical prompts to simulate non-expert attackers using publicly available tools. The models were instructed to generate biographical data about a company executive and then craft a targeted email that would appeal to the individual. Every model complied once prompt wording avoided explicit phishing terminology, demonstrating how attackers can bypass safety mechanisms with simple linguistic substitutions.

The generated emails were evaluated for accuracy, completeness, professionalism, persuasive impact and security awareness. Across the board, the models showed the ability to build credible messages that could easily bypass a target’s basic scrutiny. Some models demonstrated particularly strong structuring and persuasive framing, introducing emotional cues designed to push the recipient toward quick action. The authors emphasize that even users with little understanding of cybersecurity could create these messages by relying solely on default model settings.

The second experiment tested AI-assisted vishing, an attack that uses fraudulent voice communication to manipulate victims. The researchers followed the structure of the CFO deepfake incident and produced cloned voices using readily accessible AI voice synthesis tools. They analyzed the real and cloned voices using spectral and fingerprint comparison methods, finding that the fake audio closely matched the real in tone, pitch and cadence to the point where detection by an untrained human listener would be extremely unlikely. By adjusting specific settings, such as boosting vocal clarity, the cloned voices became even more convincing. The study shows that attackers no longer need professional audio skills or specialized hardware; the technology required for high-quality voice cloning is now democratized.

The third attack vector studied is the malicious use of AI chatbots. The researchers created a custom conversational agent designed to impersonate customer support and guide users through what appeared to be a routine account recovery process. The chatbot was trained on material from social engineering training guides, cybersecurity manuals, academic research and practical “human hacking” resources. During testing, the bot successfully built rapport with users, gradually requested sensitive information and ultimately persuaded them to provide their account password. It did this by mirroring legitimate support behavior, offering reassurance and avoiding overtly suspicious requests at the beginning of the interaction. This experiment highlights a worrying shift: AI not only makes text and audio attacks more convincing but can also sustain long, human-like conversations that lead victims toward disclosure through subtle psychological manipulation.

Across these experiments, the researchers conclude that AI does not merely automate social engineering; it enhances it by giving attackers adaptive, context-aware tools that operate at human speed and with machine efficiency.

How leading AI models perform in spear phishing scenarios

The researchers compared several leading LLMs based on two core tasks: collecting target data and generating a tailored phishing email. These tasks simulate the reconnaissance and weaponization stages of a typical spear-phishing operation.

The evaluation framework included criteria such as factual accuracy, richness of details gathered, appropriateness of cited sources, clarity of structure, professionalism in tone, persuasive strength and the presence or absence of embedded safety warnings. The results highlight key differences among the models.

Some models produced extremely detailed biographical data, pulling information from credible sources but occasionally inventing or embellishing unavailable data. This raises a dual risk: attackers can exploit hallucinated details to create narratives that feel personalized even when facts are partially inaccurate, and victims may perceive those details as legitimate due to the authoritative writing style of the message.

When assessed on generating phishing emails, the models varied in their handling of emotional triggers. Several tools framed messages using urgency, confidentiality and professional courtesy, exploiting known psychological tendencies such as fear of missing out and desire to comply with authority. None of the models inserted security disclaimers once the prompts were rephrased to avoid flagged terminology. The authors also report that all models adapted well to the revised prompts, illustrating how easy it is to circumvent built-in safety filters by simply avoiding specific keywords.

The aggregated comparison shows that while some models deliver more polished content than others, every one of the tested systems can produce material that meets the needs of a real attacker. The ranking framework developed in this study offers insights into how attackers may choose between different tools depending on desired output style, persuasive intensity or narrative depth.

The researchers stress that these findings will evolve rapidly as models are updated, and continuous testing is necessary to track how LLM capabilities shift over time. What is clear today is that no LLM, regardless of its protective measures, can be assumed safe against misuse when framed with cleverly designed prompts.

Why AI-enhanced attacks gain technical and psychological advantage

Lastly, the research focuses on the technical and psychological benefits attackers gain by integrating AI into social engineering. On the technical side, AI accelerates every phase of an attack. Text generation that once took minutes or hours now happens instantly. Voice clones can be produced without specialized skills, and chatbot-driven interactions can scale to many simultaneous victims. Attackers can run multiple experiments, refine their strategies and adapt their content in real time.

Psychologically, AI is particularly powerful because it mimics the subtle cues humans rely on in digital interactions. Whether an email uses appropriate phrasing, whether a voice sounds familiar or whether a support representative appears empathetic all play into a victim’s trust response. AI excels at reproducing these signals. In the chatbot experiment, for example, the bot used supportive language, patience and procedural familiarity to make the user feel safe. It also mirrored best practices outlined in social engineering literature: build rapport, gather baseline information, validate identity through controlled questions and only request high-value information once trust is established.

This combination of speed, scale, personalization and emotional calibration makes AI-driven attacks far harder to detect. The authors warn that as AI systems continue to improve, the gap between synthetic and authentic human communication will narrow further. This raises concerns not just for enterprises but for public institutions and individuals who may struggle to distinguish between legitimate digital interactions and malicious ones.