Phishing emails are evolving - AI may be the answer to stopping them

Phishing remains one of the most persistent cybersecurity threats, with cybercriminals constantly evolving their tactics to bypass traditional security measures. From impersonating trusted brands to crafting convincing email messages, phishing attacks exploit human psychology and technical loopholes to steal sensitive information. Despite years of research into machine learning (ML) and deep learning (DL) algorithms for email filtering, these systems still struggle with the ever-changing nature of phishing attacks. However, Large Language Models (LLMs), such as OpenAI’s GPT-4, offer new potential in detecting phishing emails with a higher level of contextual understanding and adaptability.

A recent study, “Enhancing Phishing Email Identification with Large Language Models”, authored by Catherine Lee from Georgia Institute of Technology, explores the effectiveness of LLMs in identifying phishing emails. Their work, submitted in arXiv, examines how LLMs outperform conventional ML models by considering the context, tone, and linguistic nuances of emails, rather than relying solely on rule-based detection or pattern recognition. By leveraging prompt engineering and hybrid feature selection, this study evaluates the accuracy of multiple LLMs in detecting phishing attempts while also identifying the limitations and challenges in current LLM-based detection systems.

How LLMs improve phishing email detection

Traditional phishing detection methods rely on rule-based systems and machine learning models trained on specific email characteristics, such as sender addresses, embedded URLs, and known phishing keywords. While these methods have been effective to some extent, they often struggle with new and previously unseen phishing techniques. Attackers frequently modify email structures, change domain names, and use sophisticated social engineering tactics that bypass conventional filters.

LLMs, however, introduce a more advanced approach by analyzing emails holistically. Unlike older models that focus purely on surface-level text patterns, LLMs can understand the semantic intent, tone, and structure of an email. This allows them to detect phishing attempts even when an email appears legitimate but contains subtle red flags. The study highlights that LLMs can evaluate multiple features simultaneously, including:

• Email Body Content: LLMs analyze the overall message, identifying suspicious wording, urgency cues, and manipulation techniques used in phishing emails.
• Email Subject Content: Phishing emails often contain subject lines that invoke fear or urgency, such as “Immediate Action Required” or “Security Alert.” LLMs can flag such subject lines as high-risk.
• Sender Information: Suspicious sender addresses that attempt to mimic official domains can be detected more effectively with LLM-based analysis.
• Embedded URLs: By examining hyperlink structures and inconsistencies between display text and actual URLs, LLMs can identify deceptive links commonly used in phishing attacks.

The study also compares the performance of four state-of-the-art LLMs: Llama-3.1-70b, Gemma2-9b, Llama-3-8b, and Mistral-large-latest. These models were tested on a diverse dataset consisting of spear phishing, traditional phishing, and generative AI (GenAI)-crafted phishing emails. The findings show that LLMs achieve significantly higher accuracy than previous machine learning models, particularly when dealing with complex phishing attempts.

Performance and accuracy of LLMs in phishing detection

The study evaluates the effectiveness of LLMs in phishing detection using multiple classification metrics, including accuracy, precision, recall, and F1-score. The results indicate that all four LLMs achieved over 80% accuracy, but some models performed better than others in minimizing false positives and negatives.

Among the tested models, Llama-3.1-70b emerged as the most reliable, achieving an accuracy rate of 97.21%, with a precision score of 98.10%. It also had the lowest false positive rate (4.7%), meaning it was less likely to incorrectly classify legitimate emails as phishing. This is a crucial factor in maintaining trust in automated security systems.

Other models, such as Gemma2-9b and Llama-3-8b, also performed well, with accuracy rates of 95.29% and 92.39%, respectively. However, Gemma2-9b exhibited a higher false positive rate (15%), indicating that it was more aggressive in flagging emails as phishing, which could lead to disruptions in legitimate email communications. Mistral-large-latest, while still performing reasonably well with an accuracy of 87.95%, struggled with higher false negative rates, meaning it sometimes failed to identify phishing emails.

The study also explores LLM reliability scores, which measure how consistently an LLM makes correct predictions. Llama-3.1-70b and Gemma2-9b had the highest reliability scores of 96.35% and 93.66%, respectively, making them the most trustworthy models for real-world implementation.

Challenges and limitations of LLM-based phishing detection

Despite their impressive performance, LLMs still face several challenges in phishing email detection. One key limitation is the inability to recognize personalized spear-phishing emails. Unlike mass phishing campaigns, spear phishing emails are highly customized, often addressing the recipient by name, referencing their organization, and using context-specific details to gain trust. LLMs occasionally misclassify these emails as legitimate due to their professional tone and structure.

Another limitation is the difficulty in detecting phishing attempts that rely on shortened or obfuscated URLs. Phishers frequently use URL shorteners (e.g., bit.ly) or encoded domain names to hide malicious links. While LLMs can analyze basic URL inconsistencies, they often struggle with hidden redirections or hex-encoded links, making them susceptible to false negatives.

Moreover, false positives remain a concern, particularly in emails with uncommon domain names or industry-specific jargon. Some legitimate emails, such as password reset notifications or urgent security alerts from lesser-known organizations, were mistakenly flagged as phishing. This could lead to unnecessary disruptions for businesses relying on automated email filtering systems.

To address these issues, the study suggests integrating additional cybersecurity measures alongside LLMs, such as:

• An advanced URL verification layer, using external APIs like VirusTotal to check suspicious links.
• Fine-tuned training on phishing-specific datasets, including more sophisticated social engineering tactics.
• Human-in-the-loop verification, where flagged emails undergo an additional review by security analysts before being automatically blocked.

Future directions for LLM integration in cybersecurity

The study concludes that LLMs have significant potential to enhance cybersecurity defenses, particularly in combating phishing attacks. Their ability to understand language context and social engineering techniques gives them an advantage over traditional ML models. However, they should not be solely relied upon for email security. Instead, a hybrid approach combining LLMs with existing cybersecurity frameworks can maximize their effectiveness.

One promising direction is integrating LLM-based phishing detection into corporate email systems, such as Microsoft Defender, Google Workspace, or enterprise security platforms. By combining LLMs with existing anti-phishing protocols, businesses can benefit from real-time detection, improved accuracy, and more adaptive security mechanisms.

Additionally, as generative AI phishing attacks become more prevalent, it will be essential to develop LLM-powered adversarial training techniques, where models are trained on AI-generated phishing attempts to better recognize deepfake emails and automated phishing campaigns.

With cybercriminals continuously refining their attack methods, cybersecurity professionals must stay ahead by leveraging the power of AI and LLMs. This research demonstrates that while LLMs alone cannot eliminate phishing threats, they represent a significant advancement in the ongoing fight against email-based cybercrime.