Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

The growing threat of AI in social engineering: How business can mitigate risks

AI social engineering is just getting started. Employers must make employees aware of these risks and train them to exercise their security intuition. Here's how.

[Images: happy Wu / Adobe Stock]

Profile photo of Stu Sjouwerman

Social engineering is by far the cyber industry’s most pervasive threat.

According to Verizon’s data breach report, a full three-quarters of data breaches in the last year (74%) involved the human element. In all likelihood, this number is much higher if one analyzes the root causes behind online scams, ransomware attacks, credential theft, MFA bypass, RDP hijacking, APT attacks, and others. Cybercriminals amassed about $50 billion from business email compromise (BEC) scams alone—a tiny fraction of social engineering fraud.  

WHAT IS GENERATIVE AI AND HOW DOES IT IMPACT SOCIAL ENGINEERING?

Generative AI technology such as ChatGPT has extensive business use cases—it can write content, clean up text, conduct research, help identify target audiences, respond to emails, emulate a certain style of writing, and translate text, among myriad other things.

But what if threat actors abuse these capabilities to create highly convincing, targeted, and automated phishing messages at scale? This is exactly what’s happening. As soon as ChatGPT was launched, researchers reported more than a 1000% jump in phishing emails. 

USE CASES OF GENERATIVE AI IN SOCIAL ENGINEERING

GenAI has multiple use cases in social engineering. Listed below are some recent ones:

1. Creating Highly Persuasive, Convincing, And Targeted Phishing Attacks

Traditional phishing messages have some obvious red flags—they contain spelling errors, grammatical mistakes, or generic salutations. Some victims can detect phishing messages because they encounter an unfamiliar writing style, unusual timing, or unexpected mode of communication.

Everything changes with GenAI. Attackers can instruct AI to design messages that are grammatically perfect, mimic someone’s writing style, spoof a voice, or generate a mock video.

2. Using Deepfakes To Deceive And Dupe Targets

Deepfakes are nothing but synthetic audio, video, and images that are designed for deception. Remember how young the actor Harrison Ford looked in the latest Indiana Jones movie? This same technology is freely available on mobile phone apps for anyone to weaponize for malicious purposes.

Recently, a finance worker at a multinational firm was tricked into transferring $25 million to fraudsters. Initially, the victim was suspicious. However, on a video call, he recognized people who looked or sounded like his colleagues. Similar incidents are happening worldwide, showing how AI is supercharging social engineering attacks.

3. Conducting Reconnaissance And Building Target Lists

AI can quickly assimilate and analyze large data sets on demographics, work histories, social media activity, health records, and password leaks. The resulting information is then used to build target personas based on specific demographics, occupations, interests, income range, and activity. Social engineers, state-sponsored threat actors, and APT (advanced persistent threat) actors can launch hyper-personalized social engineering and misinformation campaigns at scale.

HOW CAN ORGANIZATIONS MITIGATE AI SOCIAL ENGINEERING?

Most social engineering attacks are difficult to detect. With AI thrown in the mix, social engineering attacks are set to become far more dangerous and deceptive.

Here are some best practices that can help organizations reduce AI social engineering risks.

1. Develop Security Intuition In Employees

Social engineering works on human manipulation—not technology manipulation. Therefore, the way to manage the social engineering problem lies in strengthening security instincts.

Using regular training, communications, phishing simulation tests, and security best practices, organizations can train employees to follow their intuition to detect anomalies, develop awareness, and recognize a social engineering scam. 

2. Update Policies And Processes To Reflect AI Risks

Have clear documentation, policies, and processes in place to remind and reinforce employees of the need to stay vigilant when online. Explain how AI is evolving and how it can be used to create phishing scripts. Specify the dos and don’ts (for example: pause and think before you click or reply), explain that they must always verify the authenticity of requests, especially when large transactions are involved. Report suspicious activity to security teams and define protocols in the event of a threat or breach. 

3. Leverage Advanced Cybersecurity Tools

While social engineering attacks cannot alwaysbe stopped, they can certainly be blocked to a degree. Organizations can use phishing-resistant MFA to block an attack, even when threat actors have access to user credentials. Adopt zero trust security to reduce risk of lateral movement. Security teams can deploy email authentication protocols like SPF, DKIM, and DMARC to block email spoofing attacks.

Organizations can also adopt AI-based cybersecurity controls that can analyze large data sets and detect social engineering attempts based on contextual information such as location, IP address, and identity. Password managers should be issued to employees to reduce risk of password reuse. Security teams can also run OSINT on the organization and its people to identify potential exposures. 

AI social engineering is just getting started. Employers must make employees aware of these risks and train them to exercise their security intuition. In parallel, it’s also advisable to have multi-layered cybersecurity defenses in place so that even if one layer fails, the other can detect and block the threat. Always be prepared for the unexpected. In case of a security incident, both the organization and its employees must move quickly, without confusion, to mitigate the impact and recover from the attack.


Stu Sjouwerman is the Founder and CEO of KnowBe4 Inc., the world’s largest Security Awareness Training and Simulated Phishing platform.