AI-Powered Phishing and Deepfakes: The New Social Engineering

AI erased the typos that used to give phishing away and made voice impersonation cheap. Here is how to defend against the new generation of social engineering.

For years, the standard advice for spotting a scam email was to look for the tells: broken grammar, odd phrasing, misspelled company names. That advice is now largely obsolete. Generative AI produces fluent, well-formatted, professional messages in any language, which means the typo is no longer your early warning system. The attacker sounds just like a colleague, a vendor, or an executive.

The techniques are not new. Social engineering has always relied on trust and urgency. What is new is the quality and scale AI brings to it, and that changes what your defenses need to look like.

Why the old advice stopped working

When phishing was easy to spot, awareness training could lean on surface clues. Now a fraudulent message can be indistinguishable from a legitimate one on its face. A convincing invoice, a plausible request from a manager, a follow-up that references real projects: all of these are within reach of anyone using widely available tools.

This means detection can no longer depend on how a message looks. It has to depend on how the request is verified.

Voice cloning and executive impersonation

The more serious escalation is audio. With a short sample of someone speaking, an attacker can produce a synthetic voice that sounds like a named executive calling with an urgent instruction. The classic target is money movement: a rushed request to change bank details, approve a wire, or pay a supposedly overdue invoice before the end of the day.

The pressure is deliberate. Urgency is designed to push a staff member past the step where they would normally pause and confirm. The defense, therefore, is to make that pause mandatory rather than optional.

Verification callbacks over trusted channels

The most reliable protection against impersonation is out-of-band verification. If a request arrives by email or phone, confirm it through a different channel using contact details you already have, not the ones supplied in the message.

• Call back on a known internal number, never a number provided in the request itself.
• Confirm any payment or bank-detail change with a second, pre-established contact.
• Treat urgency and secrecy as warning signs rather than reasons to move faster.
• Give staff explicit permission to slow down and verify, even when the request appears to come from leadership.

Payment-change controls

Most wire fraud runs through a change in payment details, so that is where a hard control pays off. Require that any change to a vendor bank account or payment instruction be verified through an independent channel and approved by more than one person. This single control neutralizes the majority of these schemes because it removes the single point of failure the attacker is counting on.

Document the process so it is followed consistently, not just when someone happens to feel suspicious.

Refresh your awareness training

Training built around spotting typos needs updating. The message now is that a request can look and sound perfect and still be fraudulent, so the safeguard is the verification step, not the gut feeling. Keep it practical and repeat it, because these threats evolve.

Refresh the material to cover realistic voice and video impersonation, rehearse the callback procedure, and make sure everyone who can move money knows the payment-change rules by heart.

Practical next steps

Start with the controls that block the highest-consequence attacks: formalize a payment-change verification procedure, establish out-of-band callback habits, and refresh awareness training to reflect how convincing modern impersonation has become. A managed services partner can help you sequence this work and align it with your existing security controls.