Table of Contents
The phishing email used to give itself away: broken grammar, a generic greeting, clumsy formatting. Large language models have removed most of those tells, and they have made the expensive kind of phishing – researched, personalized spear phishing – cheap enough to send to everyone. This page looks at what the research actually shows about AI phishing, how attackers use it, and which defenses still work once you can no longer spot a fake by reading it. It is part of our coverage of password security trends and phishing.
The Evidence
What the Research Actually Shows
The most-cited figures come from a 2024 study by Fred Heiding, Simon Lermen, Andrew Kao, Bruce Schneier and Arun Vishwanath, Evaluating Large Language Models’ Capability to Launch Fully Automated Spear Phishing Campaigns. The researchers sent phishing emails to 101 volunteers recruited around a university and measured how many clicked the link:
| Click-through rate | |
|---|---|
| Control: an arbitrary phishing email | 12% |
| Written by human experts | 54% |
| Fully automated by AI (research and writing) | 54% |
| Written by AI, checked by a human | 56% |
Two things follow from this. First, the “AI phishing gets four times the clicks” claim you will see repeated is roughly what the study found – but the comparison is against a generic phishing email (54% vs 12%, about 4.5 times), not against skilled human attackers. Against the human experts, the AI did as well, not better. Second, this is one small study run in an academic setting. Treat the exact percentages as an illustration of what personalization does, not as a forecast for your inbox.
Why It Matters: The Economics
The worrying part is not that AI writes better phishing than the best human attackers. It is that it writes phishing as good as theirs for almost nothing. In the same study:
- The AI’s automated research on each target (a profile built from public sources) was accurate and useful in 88% of cases.
- A fully automated email cost roughly four cents in API fees.
- With a human checking the AI’s work, preparing each target took about 92% less time than researching and writing the email by hand.
Spear phishing used to be reserved for high-value targets because every message took real effort. When that effort drops to a few cents, everyone gets the quality of attack that was once reserved for executives. Attackers do not need better emails to succeed more often – just far more of the good ones.
What Makes AI Phishing Different
Elimination of Traditional Red Flags
For years, security awareness training taught people to look for specific signals: grammatical errors, unusual formatting, generic greetings, mismatched sender addresses. These heuristics worked because human-crafted phishing at scale inevitably contained these mistakes.
Large language models remove most of them. AI-generated text is grammatically clean, idiomatic and stylistically consistent. There are no spelling errors to catch and no “Dear Valued Customer” to trigger suspicion. Worse, the old advice can now mislead: someone who scans an email for grammar mistakes, finds none, and concludes the message is genuine has been given false confidence by outdated training.
Personalization at Scale
Traditional phishing was generic because personalization required manual research that did not scale. AI changes this. A language model can process publicly available information about a target – LinkedIn profile, social media posts, published articles, company news, press releases – and generate a personalized message in seconds. It can work out who a person’s colleagues are, what they are working on, and what kind of message they would expect to receive. The message can reference:
- The target’s actual colleagues and manager
- Recent company announcements or industry events
- The target’s professional interests and recent activities
- Contextually appropriate urgency triggers
This is not generic phishing with a name inserted. It is individually crafted social engineering at scale.
Mimicking Writing Styles
If an attacker has samples of a person’s writing – from social media, blog posts, professional articles, or emails leaked in data breaches – AI can imitate that style. The resulting email does not just come from a plausible address; it sounds like the right person. This is particularly dangerous in business email compromise, where employees are used to a specific executive’s way of writing.
Contextual Sophistication
AI models can craft phishing messages that:
- Match the communication style and tone of the impersonated sender
- Reference real events (a board meeting that was just announced, a product launch, a regulatory filing)
- Create plausible scenarios that align with the target’s role and responsibilities
- Mimic the email format and signature style of the organization being spoofed
When the message looks exactly like something your colleague would write about something that is actually happening, the usual suspicion triggers do not fire.
Real-Time Conversation
Chatbots can hold long, coherent conversations. That turns phishing from a single message into an interactive manipulation: the attacker’s bot answers replies, handles objections, and adapts to how the victim responds. This is especially effective in long-running schemes such as romance scams and tech support scams, where trust is built over many exchanges.
Multi-Language Capability
Phishing campaigns against non-English speakers used to be easy to spot because they were poorly translated. AI produces natural-sounding text in dozens of languages, opening up populations that were previously less targeted because of the attacker’s language skills.
Voice and Video
AI’s impact extends beyond text. Voice cloning can imitate a person from a short audio sample, and in December 2024 the FBI warned that criminals use AI-generated audio of a loved one’s voice to fake emergencies and demand money. “I heard it from them directly” is no longer reliable verification. Deepfake-as-a-service tools put the same capability within reach of low-skill attackers.
The Attack Chain
AI phishing is most dangerous when understood as part of a larger attack chain rather than an isolated technique:
Reconnaissance: AI processes open-source intelligence about the target – social media, professional profiles, corporate filings, data broker records.
Message generation: The language model crafts a personalized, contextually appropriate message for that specific person.
Delivery: The message is sent through channels that maximize credibility – spoofed or lookalike addresses, compromised legitimate accounts, or messaging platforms.
Credential capture: The target clicks through to a convincing replica of a real login page and enters their credentials, or opens an attachment that installs an infostealer.
Reuse: Captured credentials are used immediately and tried on other services through credential stuffing. The compromised account may be used to phish the target’s contacts, creating a trusted-sender chain.
Monetization: Compromised accounts are exploited for financial fraud, data theft, ransomware deployment, or further credential harvesting.
AI lowers the attacker’s cost at nearly every step.
Why Traditional Defenses Fall Short
Security Awareness Training
Training built around spotting red flags in the message itself is losing ground when the message contains none. This does not make awareness training useless – knowing that phishing exists and keeping a baseline of skepticism toward unexpected requests still matters. But the specific heuristics taught in most programs (check for spelling errors, look for generic greetings) are increasingly inadequate.
Email Filters
Email security gateways use sender reputation, URL analysis, content heuristics and machine learning classifiers. AI-generated phishing challenges them because:
- Every message can be unique, so it does not match known templates
- The language is indistinguishable from legitimate email
- Compromised legitimate accounts have good sending reputations
Filters are adapting with their own AI models, but the defender has to be right every time while the attacker only needs to succeed once.
URL and Domain Inspection
Telling people to inspect URLs before clicking is increasingly impractical. Modern phishing uses lookalike domains, URL shorteners and redirects, and on phones – where much email is read – the address bar is small and truncated.
What Actually Works
Passkeys and Phishing-Resistant Authentication
The most effective defense against AI phishing is architectural, not behavioral. Passkeys are bound to the legitimate website’s domain. Even if you click a phishing link and land on a perfect replica of a login page, the passkey will not work there, because the browser knows the domain is wrong. No amount of AI polish changes that: the check happens in the protocol, not in your judgment. The remaining weak spot is any fallback you leave enabled, such as a password or SMS code that can still be phished.
Password Managers with Autofill
Password managers provide strong protection through domain-matched autofill. The manager compares the current site’s domain with the one saved for your login. On a phishing site, it does not offer to fill your credentials – and that silence is your warning. See how a password manager protects you from phishing for the details.
This protection only works if you use autofill rather than typing passwords by hand. PanicVault and other managers that integrate with system AutoFill on Apple devices provide it across Safari and apps that use the credential provider framework.
Zero Trust Verification
Since you can no longer rely on the quality of the writing, judge the context instead:
- Is this request expected? Did you anticipate it, or did it arrive out of nowhere?
- Is it normal? Does it match how this person or organization usually deals with you?
- Is there urgency? Manufactured urgency remains the most reliable phishing signal, however polished the text.
- Are you being asked to bypass a procedure? Any request to skip normal processes – even from someone senior – deserves extra scrutiny.
Then adopt a zero trust approach to anything that asks you to enter credentials, open a file, move money or change account settings: verify it through a separate channel you already trust. If your boss emails with something urgent, call them on a number you already have, not one from the email. This out-of-band verification defeats phishing regardless of how convincing the message is.
Credential Hygiene
Even when phishing succeeds, good credential practices contain the damage:
- Unique passwords everywhere: A password stolen by phishing gives the attacker one account, not your entire digital life.
- Two-factor authentication: Even with a stolen password, the attacker needs the second factor – though real-time phishing proxies can relay one-time codes, which is why passkeys are stronger.
- Breach alerts: Services that warn you when your credentials appear in a breach give you time to change them before they are exploited.
PanicVault and the KeePass ecosystem make credential hygiene practical by generating unique, random passwords for every account and storing them in an encrypted, portable database. The open KDBX format means your credentials are never locked into a single vendor.
Accept Uncertainty
The most important mindset shift is to accept that you cannot always tell whether a message is genuine just by looking at it. That is not a personal failing; it is the reality of AI-generated content. Instead of trying to be a perfect detector, build systems and habits that protect you even when a phishing message fools you:
- A password manager that will not autofill on the wrong domain
- Strong, unique passwords for every account, so a single compromise does not cascade
- Passkeys or two-factor authentication on every account that supports them
- Verification habits that have become automatic
Training humans to outperform AI at spotting fakes is a losing strategy. Defenses that work independently of human judgment – passkeys that cannot be phished, password managers that check domains, and out-of-band verification for sensitive requests – are the ones that hold up.
