AI Phishing: How Fake Emails, Fake Voices, and Fake Vendors Are Changing Cyber Risk

AI has not reinvented phishing so much as removed many of the weaknesses that once made it easier to spot. Fake emails are cleaner, voice clones can sound convincing, vendor impersonation is easier to personalise, and attackers can now produce believable social-engineering campaigns at a scale that once required far more time and research.

TL;DR

AI is changing phishing because it reduces the cost of credibility. Attackers can produce better-written emails, imitate executives or suppliers, generate convincing voice messages, translate scams into multiple languages and personalise attacks using information gathered from public sources. What used to require careful manual research can increasingly be automated, allowing attackers to create more believable lures for far more people.

The larger risk is that phishing is moving beyond the inbox. A fraudulent payment request may arrive as a voice message from someone who sounds like the CFO, a supplier may appear to join a video call, or an employee may receive a highly personalised message referencing a real project or vendor relationship.

Companies therefore need to stop treating authenticity as something employees can judge from appearance or tone alone and start building verification into the workflows where money, credentials, access and sensitive information can change hands.

Key Takeaways

• AI makes phishing cheaper to personalise, which allows attackers to create convincing messages for far larger groups without the manual research traditionally associated with spear phishing.
• The biggest change is not simply better grammar. AI can now imitate voices, generate video, create synthetic identities and reproduce the tone of executives, suppliers and colleagues.
• Business email compromise becomes more dangerous when a fake email can be reinforced by a convincing voice note, WhatsApp message or video meeting.
• Vendor impersonation deserves particular attention because attackers can exploit real invoices, payment cycles, procurement relationships and compromised email threads rather than inventing an entirely fictional story.
• Employees can no longer rely on familiar voices, professional writing or realistic video as proof of identity. High-risk requests need independent verification.
• The most effective defence is procedural as much as technical. Payment changes, account recovery, privileged access and sensitive data requests should require verification that does not depend on the communication channel being attacked.

When Seeing and Hearing Are No Longer Proof

In early 2024, an employee at engineering firm Arup joined what appeared to be a perfectly legitimate video conference with senior colleagues. The people on screen looked and sounded familiar, and the request itself appeared to come from management.

Over the course of the interaction, the employee authorised 15 transfers to five bank accounts, ultimately sending around HK$200 million, roughly £20 million at the time, before discovering that the people in the meeting were deepfake recreations rather than colleagues.

Arup later confirmed that fake voices and images had been used in the fraud, while Hong Kong authorities subsequently recorded the case among a small but growing number of frauds involving deepfake technology.

This incident shows what has actually changed. Phishing has always depended on impersonation, but the attacker traditionally had to leave clues behind. The email might contain awkward language, the phone call might not sound quite right, or an unexpected request from a senior executive could be checked by calling them directly.

Generative AI is steadily removing those imperfections. It can write fluent business correspondence, reproduce a person’s speaking style, generate synthetic audio from relatively limited samples and create video convincing enough to survive at least a short business interaction.

The FBI now treats this as an increasingly practical fraud technique rather than a theoretical future risk. Its 2025 Internet Crime Report recorded more than 22,000 complaints involving AI and adjusted losses exceeding $893 million, including more than $30 million in reported business email compromise losses where AI was involved.

The same report notes that chat generators can produce official-looking messages impersonating executives, while voice cloning can be used to request payments or pose as employees. Those numbers do not mean every AI-related loss began with phishing, but they show how quickly synthetic content has entered ordinary fraud operations.

The more important shift is economic. Traditional spear phishing was expensive because good impersonation took time. An attacker targeting a CFO might have to study LinkedIn, read company announcements, identify suppliers, understand reporting lines and manually craft a plausible message.

AI does not remove that research requirement entirely, but it can compress the work dramatically. Once the attacker has enough context, the same information can be turned into multiple personalised emails, translated versions, voice scripts, fake supplier communications and follow-up conversations without requiring a person to write every variation from scratch.

Microsoft has already described this combination of scale and credibility in the wild. During its 2026 disruption of a cybercrime infrastructure service called RedVDS, Microsoft said attackers were pairing rented infrastructure with generative AI to identify valuable targets and create more realistic multimedia email conversations. In some cases, those campaigns were supplemented with face swapping, manipulated video and voice cloning.

Microsoft observed more than 2,600 RedVDS virtual machines sending an average of roughly one million phishing messages per day to Microsoft customers during one month alone, which illustrates what happens when convincing social engineering becomes easier to manufacture at industrial scale.

The result is that several signals employees once used as shortcuts for authenticity are becoming weaker at the same time. Good grammar is no longer reassuring. A familiar voice is not necessarily proof of identity. A video call can no longer be treated as absolute confirmation that the person on screen is real. Even a supplier who appears to know invoice numbers, project details or internal terminology may be working from information taken from a compromised mailbox or public sources.

WPP faced exactly this kind of multi-channel impersonation in 2024 when fraudsters targeted chief executive Mark Read. The attackers created a fake WhatsApp account using a publicly available image, arranged a Microsoft Teams meeting and used an AI-generated voice clone alongside video footage to impersonate senior executives.

The attack was unsuccessful, but its structure is more revealing than the outcome because the criminals did not rely on one fake email. They built several reinforcing pieces of evidence so that each channel made the others appear more credible.

This is where AI phishing becomes a different operational problem. The company is no longer defending only against a malicious message that looks suspicious when inspected carefully. It may be defending against an email that reads correctly, a caller who sounds familiar, a video meeting populated by recognisable faces and a vendor request containing accurate business information. None of those signals can be discarded entirely, but none of them should be sufficient evidence for a sensitive decision either.

The security question therefore changes from “Can our employees spot AI-generated content?” to “Which business actions still depend on someone deciding that a message, voice or face looks genuine?”

Payment changes, privileged account recovery, new vendor instructions, executive requests and transfers of sensitive information are where this distinction matters most, because AI does not need to fool everyone in the company. It only needs to make one high-consequence request believable enough for the right person to act on it.

The New Attack Surface Is Trust Itself

AI phishing becomes more dangerous when it stops looking like “phishing” at all. The strongest attacks do not necessarily arrive as suspicious emails from unknown senders. They arrive through the signals people normally use to verify legitimacy: a familiar voice, a recognisable face, a known supplier, a correctly written message, or a conversation that appears to continue an existing business relationship.

This is why deepfake fraud matters far beyond sensational video clips. In a traditional phishing attempt, the attacker has to persuade the victim that a message is genuine. In an AI-assisted attack, the attacker can manufacture several pieces of supporting evidence at once.

An email can match an executive’s writing style, a voice note can sound like the same person, and a video call can reinforce the illusion. Each element reduces the likelihood that the recipient will challenge the others.

The 2024 WPP impersonation attempt is a good example because the attackers combined channels rather than relying on one fake message. Fraudsters created a WhatsApp identity using a public image of CEO Mark Read, arranged a Microsoft Teams call, and used AI-generated voice and manipulated video to impersonate senior executives.

The attack ultimately failed, but the method shows where social engineering is heading: not towards a perfect fake in one channel, but towards a collection of believable signals that support each other.

Vendor fraud presents an even more practical risk because the attacker may not need to invent the relationship at all. If a supplier mailbox has been compromised, criminals can study invoice patterns, payment dates, project names and the language used between both companies.

AI can then help turn that context into more convincing follow-up communication, while synthetic audio gives the attacker another way to “confirm” a fraudulent bank-account change if the finance team calls back through a compromised or attacker-controlled channel.

The weakness here is that many companies still verify identity using the same channel through which the request arrived. A suspicious email is confirmed by replying to the email. A payment change is checked through the phone number included in the message. A voice request is trusted because the caller sounds familiar. Once AI can imitate those signals, verification has to move outside the attacker’s control.

A useful way to think about this is:

Old assumption	AI-era problem
Good grammar suggests legitimacy	AI can produce polished business writing instantly
A familiar voice confirms identity	Voice cloning can imitate colleagues and executives
Video proves the person is real	Deepfake video can simulate a credible meeting
Vendor knowledge suggests authenticity	Compromised mailboxes can expose real commercial context
Calling back confirms the request	The number or channel itself may be attacker-controlled
Multiple matching signals increase confidence	AI can manufacture several matching signals together

The practical implication is that sensitive actions need verification based on something the attacker cannot easily reproduce from public information, compromised communications or synthetic media. A vendor bank change should be checked against previously verified contact details.

A high-value executive request should require a second approver or an independent internal channel. A helpdesk reset should depend on stronger identity proofing than voice familiarity or knowledge-based questions.

The larger point is that AI phishing attacks the shortcuts people use to establish trust. Once appearance, voice, writing style and familiarity become reproducible, the organisation has to move from “Does this look genuine?” to “What independent evidence do we require before this action can happen?” That shift is much more important than teaching employees how to identify an AI-generated voice or spot visual glitches in a deepfake.

Vendor Impersonation Is Becoming Harder to Separate From the Real Thing

Vendor fraud has always relied on familiarity, but AI makes that familiarity easier to manufacture. A finance team may already know the supplier name, recognise the usual invoice format and expect a payment request at the end of the month. If an attacker has access to compromised email threads, public company information or leaked commercial data, AI can help turn those fragments into a much more convincing imitation of the real relationship.

The danger is that the fraudulent request can arrive with details that would once have seemed reassuring. It may reference a genuine project, use the right account manager’s name, match the supplier’s usual tone and explain a bank-account change in language that feels operationally plausible.

The FBI’s guidance on business email compromise has long warned that attackers monitor real business relationships and payment patterns before attempting fraud, and generative AI now makes it easier to reproduce those patterns across more targets and more channels.

This changes the way finance teams should think about “known vendors.” A known vendor is only useful as a trust signal if the communication channel is still trustworthy. Once an email account has been compromised or a supplier identity has been convincingly reproduced, the history of the relationship can work against the company because familiarity lowers resistance.

What looks reassuring	Why it may no longer be enough
Correct supplier name	Easily gathered from invoices, websites or email threads
Familiar tone and writing style	AI can imitate prior correspondence
Accurate project references	Real context may come from compromised accounts
Correct invoice timing	Payment cycles can be observed before the attack
Voice confirmation	Synthetic audio can imitate known contacts
Video confirmation	Deepfake video can reinforce the impersonation

The most effective control is therefore not better intuition but independent verification. A request to change banking details should be confirmed using contact information already held on file, not the number or email address provided in the change request. High-value or unusual transactions should require a second approval, and procurement or finance systems should flag changes to payment instructions before money is released.

AI makes these controls more important because it reduces the value of surface-level authenticity. A polished email, a familiar voice or a convincing explanation can all be manufactured. What is harder to manufacture is a verification process that sits outside the attacker’s communication path and requires more than one person or one channel before a sensitive change is accepted.

Voice Cloning Changes the Meaning of “Call Me to Confirm”

For years, one of the simplest safeguards against suspicious email was also one of the most sensible: if the request involves money, credentials or sensitive information, call the person and confirm it.

That advice still works when the callback goes to a known number and the conversation is genuinely independent, but AI-generated speech has weakened the assumption underneath it. Hearing a familiar voice is no longer strong evidence that the familiar person is actually on the other end.

This is not limited to elaborate deepfake video calls. The FBI warned in 2025 that malicious actors were already using AI-generated voice messages to impersonate senior US officials, often beginning with a text or voice message designed to establish rapport before moving the target onto another communication platform.

The underlying technique translates easily into business fraud because an executive, supplier, customer or colleague does not need to speak for ten minutes to sound convincing. A short voice note saying, “I’m heading into a meeting, please get this processed and message me when it’s done,” may be all the attacker needs if the surrounding context already makes sense.

Voice cloning is particularly effective because people are accustomed to treating voice as a stronger identity signal than text. An unusual email may trigger suspicion, but hearing a recognisable accent, cadence or manner of speaking can resolve that suspicion rather than deepen it.

The FBI has separately warned businesses that criminals are using AI-powered voice and video cloning to impersonate co-workers and business partners, precisely because synthetic media can add another layer of apparent authenticity to requests for money or information.

The risk becomes greater when voice is used to support an attack that has already begun elsewhere. An attacker may first send a credible message from a spoofed or compromised account, then follow with a voice note that appears to come from the same executive.

Another may impersonate a supplier over email and use cloned audio when finance asks for confirmation. In both cases, the voice does not have to carry the entire fraud. It only has to resolve the remaining doubt.

Old verification habit	Why AI weakens it	Stronger alternative
“I heard their voice, so it must be them”	A short sample can be synthetically reproduced	Call a previously verified number and follow a defined approval process
Replying to a voice note	Keeps the conversation inside the attacker’s chosen channel	Start a new conversation through an established company channel
Asking personal questions	Public information may provide convincing answers	Use controls the attacker cannot research socially
Calling the number in the message	The contact detail may belong to the attacker	Use the number already stored in company records
Trusting a senior executive’s urgent instruction	Authority and synthetic voice reinforce each other	Apply the same approval threshold regardless of seniority

This also means companies need to reconsider some older fraud-prevention advice. Verification cannot simply mean adding another communication channel if the attacker controls both channels. An email followed by a WhatsApp voice note is still one unverified identity presented twice. What matters is whether the second check originates independently, using contact information and procedures established before the suspicious request appeared.

There is an important organisational point here as well. Employees should not be expected to become forensic analysts of synthetic speech, listening for robotic pauses, strange breathing or pronunciation errors before approving a transaction.

Those clues may occasionally help, but the FBI itself cautions that AI-generated voices can sound almost identical to the person being impersonated. As models improve, any defence based primarily on spotting imperfections will age badly.

The stronger approach is to make identity and authority separate questions. A caller may genuinely sound like the CEO, but that should not determine whether a new bank account can be approved, whether credentials can be reset or whether confidential data can be released. The organisation should already have rules for those actions, and those rules should survive even when the person making the request sounds completely authentic.

AI Is Turning Spear Phishing Into a Volume Business

The real shift with generative AI is not that attackers suddenly discovered personalisation. Spear phishing has been personalised for years. What has changed is the cost of doing it well.

An attacker can now take a small amount of public or stolen information about a company, feed it into an AI system, and generate dozens or hundreds of plausible variations built around different roles, projects, suppliers and internal workflows. That starts to blur the old line between broad phishing and highly targeted spear phishing.

Microsoft has already documented campaigns where this is happening in practice. In an AI-enabled device code phishing campaign observed in 2026, attackers used generative AI to create role-specific lures built around realistic business themes such as RFPs, invoices and manufacturing workflows.

Microsoft also found that the attackers enriched targets using public profiles and corporate directories, then focused follow-on activity on people in financial and executive roles. The point is not simply that AI wrote a cleaner email. It helped connect reconnaissance, personalisation and targeting into one faster attack process.

That changes the economics quite dramatically. A traditional attacker might spend meaningful time researching one executive before crafting a convincing message. AI-assisted phishing makes it much easier to repeat that process across an entire finance team, sales organisation or supplier network, with each recipient receiving something that appears relevant to their own role.

Proofpoint’s 2026 analysis similarly argues that AI is being used to personalise attacks at scale and automate large parts of the attack chain, while reporting a 94% year-on-year increase in email threats targeting its customers in 2025.

The practical difference is easier to see when the two models are compared:

Traditional broad phishing	AI-assisted targeted phishing
One generic message sent to thousands	Different versions built around recipient roles
Obvious language or formatting mistakes more common	Fluent, professional language is trivial to generate
Limited research on individual victims	Public profiles can be summarised and incorporated quickly
Manual translation slows international campaigns	Messages can be localised into multiple languages almost instantly
Reused templates create recognisable patterns	Large numbers of variants can be generated from the same campaign
Follow-up often generic	Replies and objections can be answered in context

Attackers are also becoming better at hiding inside legitimate infrastructure, which makes personalisation more convincing because the message no longer has to arrive from an obviously suspicious source.

Proofpoint documented a 2025 campaign that abused Microsoft 365’s Direct Send feature to make phishing emails appear to originate from inside the victim’s own organisation. The lures included ordinary business themes such as task reminders, wire authorisations and voicemail notifications, which is precisely the kind of material employees are conditioned to process quickly.

This matters because many traditional phishing controls were designed around repetition. If thousands of identical messages contain the same suspicious wording, URL or attachment, detection systems have something obvious to cluster.

AI-generated campaigns can produce more variation in language, subject lines and pretexts without changing the attacker’s underlying objective. Microsoft has already seen attackers use AI to obfuscate phishing payloads and generate unusual variants, even though the company was still able to detect those campaigns using infrastructure, behavioural and message-context signals.

The last point is important because AI does not make phishing invisible. It makes some of the old indicators less reliable. Security teams can no longer assume that poor writing, repeated templates or crude personalisation will expose the campaign, but they can still look at where the message came from, what infrastructure it touches, what the requested action does and whether the behaviour fits the user’s normal activity.

In other words, as attackers become better at manufacturing believable content, defenders increasingly have to judge the behaviour around the content rather than the polish of the content itself.

The Hardest AI Phishing May Look Completely Ordinary

As AI improves the quality of the message, defenders increasingly have to look beyond the message itself. The polished email is only the visible part of the attack. What matters more is where it came from, which account sent it, what the recipient is being asked to do, whether the behaviour fits the normal relationship, and what happens immediately after the interaction.

Microsoft has already documented phishing campaigns that spoof internal organisational domains by exploiting routing and email-security misconfigurations, allowing messages about password resets, HR notices, shared documents and voicemails to appear as though they originated from inside the company. When a convincing message also carries an apparently legitimate internal identity, the employee has fewer reasons to question it.

The same problem appears in industry-specific attacks. Microsoft identified an ongoing campaign targeting hospitality businesses that impersonated Booking.com and used realistic reservation-related complaints and requests. The lure worked because the recipient was not being asked to engage with something unfamiliar.

Hotel employees routinely deal with bookings, customer complaints and travel-platform notifications, so the attacker inserted malicious instructions into a workflow that already demanded regular attention. AI makes that approach easier to reproduce across sectors because the language, terminology and business scenarios can be adapted quickly to match the target.

That creates an important distinction between authenticity and normality. A request can feel normal without being authentic, and AI makes the gap between the two harder to see. Security teams therefore need to pay more attention to behavioural signals that sit around the content rather than relying heavily on whether the message itself looks professional.

What employees see	What security teams should examine
A polished supplier email	Is the sender, domain and payment behaviour consistent with previous interactions?
A familiar internal request	Did the message actually originate through the expected mail route?
A realistic cloud notification	Is the destination domain, application and requested action legitimate?
A believable invoice	Has the beneficiary, amount or payment pattern changed unexpectedly?
A convincing executive request	Is the behaviour consistent with that executive’s normal approval process?
A professional attachment or shared file	What happens when the file is opened, and does it initiate unusual execution or authentication?

This also explains why generative AI creates difficulties for older detection methods that depend heavily on recurring templates or obvious malicious code. In 2025, Microsoft described an attack in which AI-generated code appears to have been used to obfuscate a credential-phishing payload inside an SVG file.

The file was designed to resemble an ordinary shared PDF, while the underlying code contained enough unnecessary complexity and synthetic structure to make static inspection harder. The attack was still detected, but through a combination of behavioural, infrastructure and content signals rather than a simple visual clue available to the employee.

This is where the defensive advantage also starts to shift back towards automation. AI can help attackers create more variants, but defenders can analyse identity behaviour, login patterns, message routes, endpoint activity and payment anomalies at a scale no employee could realistically manage.

Microsoft’s own 2025 analysis of AI-powered deception makes the same broader point: fraud is increasingly built from combinations of deepfakes, voice cloning, convincing websites and personalised communication, which means controls need to assess the interaction as a whole rather than one suspicious-looking artefact.

For businesses, the implication is fairly practical. Employees should still be trained to question unusual requests, but organisations should not make “spot the fake” their primary AI-phishing strategy. As synthetic content improves, some fraudulent messages will look entirely professional and some voices will sound entirely familiar.

The stronger defence is to identify unusual behaviour around the request, require independent verification for high-consequence actions, and use security controls that remain effective even when the content itself looks completely genuine.

AI Is Forcing Companies to Redesign Verification, Not Just Detection

Once email, voice and video can all be convincingly imitated, companies eventually run into a limit that no amount of awareness training can solve. Employees cannot be expected to determine authenticity from presentation alone when the presentation itself can be manufactured. The more durable response is to redesign high-risk workflows so that sensitive actions depend on independent evidence, not on how convincing the person making the request appears to be.

This matters most wherever an attacker can turn trust directly into money, access or sensitive information. Payment-detail changes, payroll updates, password resets, MFA resets, privileged account recovery and requests for confidential files should all have verification steps that remain valid even if the original email, call or video is fake.

The FTC’s guidance on business impersonation scams makes the same basic point from the fraud side: when an unexpected message appears to come from a known business, people should not use the contact information supplied inside that message to verify it.

For companies, that principle needs to become operational rather than advisory.

High-risk action	Weak verification	Stronger verification
Vendor bank-detail change	Replying to the email requesting the change	Calling a previously verified supplier contact and requiring second approval
Executive payment request	Trusting voice, video or seniority	Confirming through an established internal channel and existing approval workflow
Password or MFA reset	Knowledge questions or voice recognition	Strong identity proofing and additional approval for privileged accounts
Payroll-account change	Email or chat confirmation	Verification through the authenticated HR system or previously registered contact path
Sensitive data request	Familiar sender and plausible business reason	Confirming identity, authority and business need separately
New cloud application	User accepts a convincing consent screen	Restricting app consent and requiring administrative review for risky permissions

Authentication deserves particular attention because AI phishing becomes much less valuable if stolen credentials cannot easily be converted into access. CISA recommends organisations move towards FIDO/WebAuthn because phishing-resistant authentication can reject authentication attempts made through fraudulent websites even when the employee has been persuaded to interact with them.

The important improvement is structural: instead of asking the user to decide whether the page looks genuine, the authentication protocol itself helps determine whether the request belongs to the legitimate service.

The same thinking should be applied to privileges after login. CISA’s broader hardening guidance recommends role-based access control, least privilege and tighter session management, because stopping the initial deception is only one part of the problem. If an attacker compromises an ordinary account but cannot reach finance systems, customer databases or administrative tools without additional controls, the potential impact changes considerably.

For growing companies, this does not require rebuilding every business process at once. The useful approach is to identify the small number of actions where impersonation would create the greatest consequence and make those actions difficult to complete through trust alone.

In practice, that usually means starting with finance, privileged identity, payroll, executive approvals and customer administration, then expanding the model as the organisation matures.

The larger change is conceptual. Verification can no longer mean, “I recognised the sender”, “the voice sounded right”, or even “I saw them on video”. In an environment where those signals can increasingly be generated, the strongest organisations will treat identity, authority and approval as separate things that each need to be established before a high-consequence action goes through.

What Companies Should Change Before AI Phishing Gets Better

The practical mistake would be to respond to AI phishing by buying a deepfake detector and adding another training module. Both may help, but neither addresses the core problem: many business processes still allow a convincing identity to substitute for actual verification.

If a payment can be redirected because a supplier email looks right, or a privileged reset can happen because a caller sounds like an executive, then the weakness sits in the workflow before it sits in the content.

The first priority should therefore be to identify the small number of actions where impersonation can create serious consequences and put controls around those actions.

The FBI’s current guidance on business email compromise specifically recommends independently verifying payment requests and any change in account or payment procedures rather than relying on the information supplied in the request itself. That advice becomes even more important when AI can make the email, voice message and supporting explanation all look credible at the same time.

Priority area	What should change	Why AI makes it more urgent
Vendor payments	Verify bank-detail changes using contact information already on file and require secondary approval	AI can reproduce supplier tone, invoice context and follow-up communication
Executive requests	Apply normal approval rules regardless of who appears to be asking	Voice and video impersonation make authority easier to fabricate
Identity recovery	Strengthen password and MFA reset procedures, particularly for privileged users	Social engineers can combine public information with synthetic voice or video
Authentication	Move high-risk users towards phishing-resistant authentication	Better phishing content matters less if captured credentials cannot be replayed
SaaS permissions	Restrict risky app consent and administrative privileges	A convincing consent flow can provide access without conventional password theft
Finance monitoring	Flag unusual beneficiaries, payment destinations and transaction patterns	Behaviour can expose fraud even when the communication looks authentic
Incident reporting	Make suspicious voice, video, email and vendor requests easy to escalate	Employees need somewhere to send uncertainty before acting on it

There is also a technical side to this shift. As attackers become better at generating convincing content, defenders need to rely more heavily on signals that synthetic media cannot easily disguise. Microsoft’s threat research shows that AI is now being used for dynamic phishing personalisation, synthetic identities and real-time voice manipulation, while the company’s 2025 defence report found a 195% global increase in AI-driven identity forgeries, including techniques capable of defeating some selfie and liveness checks.

That makes identity behaviour, device trust, authentication method, transaction history and access patterns increasingly valuable because they test what the user or account is doing rather than whether the communication looks convincing.

For most growing companies, however, the biggest improvements are likely to be procedural before they are exotic. Finance should know that a new vendor bank account cannot be approved from an email thread alone. IT support should know that a familiar voice does not override identity-recovery requirements.

Executives should understand that their seniority does not exempt an urgent request from normal approval controls. Employees should have a simple way to escalate something that feels wrong without first having to prove that it is fake.

The useful principle is to design high-consequence processes so that synthetic credibility is not enough. An attacker may eventually be able to reproduce the email, the voice, the face and the context almost perfectly, but they should still have to defeat an independent approval, a known contact path, a phishing-resistant authentication method or a behavioural control before the organisation gives them money, access or sensitive information.

The New Security Question Is No Longer “Does This Look Real?”

AI phishing changes the standard of proof inside a company. For years, people relied on familiar signals because they were usually good enough: the email sounded right, the supplier knew the project, the executive’s voice was recognisable, or the person appeared on video.

Those signals are now becoming easier to reproduce, and the FBI has already warned that AI-generated voice messages can sound nearly identical to the person being impersonated. Once that happens, appearance and familiarity become useful context, but poor evidence.

The scale of impersonation fraud makes this more than a theoretical concern. The FTC recorded $3.5 billion in reported losses to imposter scams in 2025, with nearly one in three fraud reports involving some form of impersonation.

AI will not account for every one of those cases, but it makes the underlying model easier to execute because attackers can generate more convincing identities, conversations and supporting material without the cost that sophisticated impersonation once required.

The practical response is not to ask employees to become better judges of synthetic media. Even the FTC has argued that the risks from AI voice cloning cannot be addressed through technology alone, which is why verification, authentication and business process matter so much.

A finance employee should not have to decide whether the CFO’s voice sounds 98% genuine before approving a transfer, just as a support analyst should not have to determine whether the person on a video call is a deepfake before resetting a privileged account.

The companies that adapt best will therefore separate credibility from authority. An email can look genuine without being authorised. A caller can sound genuine without having permission to change a payment. A supplier can know every detail of a project without being entitled to redirect an invoice.

Once high-risk actions require independent verification, established contact paths and additional approval, AI-generated realism becomes much less useful to the attacker. That is ultimately what AI phishing changes. It does not make trust irrelevant, but it makes informal trust much more expensive to rely on.

FAQs

1. What is AI phishing?

AI phishing is phishing that uses generative AI to improve some part of the attack, whether that means writing more convincing emails, creating personalised messages, cloning a voice, generating synthetic video or imitating a supplier or executive more realistically. The basic objective has not changed. The attacker still wants someone to reveal credentials, approve a payment, provide sensitive information or grant access.

What has changed is the quality and economics of impersonation. Tasks that once required manual research, strong language skills or specialist editing can now be completed much faster, which allows attackers to personalise more campaigns without sacrificing scale.

2. How can you tell if an email was written by AI?

In many cases, you cannot tell reliably from the writing alone. Good grammar, professional formatting and natural language are no longer useful indicators of legitimacy because AI can reproduce them easily, while human-written phishing can be polished as well.

A better approach is to examine the request rather than trying to diagnose the author. Unexpected payment changes, new login pages, unusual file-sharing requests, requests for credentials and departures from normal business processes deserve verification regardless of whether the message sounds human or AI-generated.

3. Can AI really clone someone’s voice well enough to fool employees?

Yes, particularly when the attacker only needs a short piece of audio to support an existing story. A voice note or brief phone conversation does not have to reproduce every detail of someone’s speech perfectly if the recipient already expects to hear from that person and the surrounding request appears plausible.

That is why voice recognition should not be treated as sufficient verification for high-risk actions. Companies should confirm unusual financial, access or data requests through established processes that do not depend on whether the caller sounds familiar.

4. Are deepfake video calls actually being used in business fraud?

Yes. Criminals have already used synthetic or manipulated video and audio in attempts to impersonate executives and colleagues during business interactions. The risk becomes particularly serious when the video call is used to reinforce another piece of communication, such as an urgent email or payment request.

The practical lesson is not that every video call should be treated as suspicious. It is that appearing on camera should not override normal controls. High-value payments, identity resets and unusual approvals should follow the same verification process whether the request arrives by email, phone or video.

5. Why are vendors such attractive targets for AI phishing?

Vendor relationships already contain many of the ingredients an attacker needs: regular invoices, known contacts, predictable payment cycles and established trust. If criminals gain access to an email account or collect enough business information, they can imitate that relationship rather than creating a completely fictional one.

AI makes the imitation easier by helping reproduce writing style, generate plausible explanations and create follow-up communication quickly. This is why changes to bank details, payment destinations or unusual invoice instructions should always be verified independently.

6. Can AI phishing bypass multi-factor authentication?

Some forms of phishing can still work around weaker MFA methods. Attackers may attempt to capture one-time codes, relay authentication in real time, manipulate push approvals or steal authenticated sessions rather than simply taking a password.

Phishing-resistant authentication provides stronger protection because it reduces the usefulness of captured credentials and fake login pages. Companies should therefore look at the type of MFA they use, particularly for finance, administrators, executives and other high-risk accounts.

7. Will AI make traditional phishing training useless?

No, but it makes generic training less valuable on its own. Employees still need to understand suspicious requests, credential theft, payment fraud and how to report incidents, but they cannot be expected to identify every AI-generated email, voice or video through visual or linguistic clues.

Training becomes more useful when it focuses on behaviour and process. Employees should know which actions require verification, where normal workflow ends, how to challenge unusual requests and what to do immediately if they believe they have been deceived.

8. What should finance teams do about AI vendor impersonation?

The strongest control is to separate payment verification from the message requesting the change. New banking details should be confirmed using contact information already held by the company, while significant changes should usually require another approver before funds are released.

Finance teams should also become suspicious of changes in behaviour rather than merely suspicious-looking emails. A familiar supplier requesting payment to a new jurisdiction, a sudden beneficiary change or unusual urgency can matter more than whether the email itself looks perfectly legitimate.

9. What is the biggest mistake companies make when defending against AI phishing?

The biggest mistake is trying to solve the problem primarily by teaching employees to spot AI-generated content. That strategy becomes weaker as synthetic content improves because the attacker is specifically trying to remove the abnormalities employees have been trained to notice.

A stronger model assumes that convincing impersonation will sometimes succeed and makes high-consequence actions difficult to complete through trust alone. Verification, access controls, approval rules, stronger authentication and rapid reporting then become part of the phishing defence rather than something that happens after it fails.

10. How should a growing company prepare for AI phishing?

Start with the workflows where impersonation could create the most damage. Vendor payments, payroll changes, executive approvals, privileged account recovery, customer administration and sensitive-data requests usually deserve attention before lower-risk processes.

The objective is not to build an elaborate AI-security programme immediately. It is to make sure that a realistic email, familiar voice or convincing video cannot by itself move money, reset a critical identity or release sensitive information. Once those controls are in place, detection tools and training become much more effective layers around them.