Recognizing Social Engineering in the Age of AI

For years, employees have been trained to recognize phishing by looking for familiar warning signs: a misspelled word, an unusual greeting, or a message that simply doesn't sound like something a colleague would write. Now, with artificial intelligence (AI) in the mix, those warning signs still have value but are becoming less reliable.

AI gives threat actors an easier way to create polished messages that resemble everyday business communications. A phishing email can reference the right company, use natural language, and sound remarkably similar to a legitimate request. Use of AI changes what employees need to focus on to recognize social engineering scams.

A convincing message can still lead to danger

Social engineering attacks succeed when an employee is persuaded to take an action that benefits the threat actor. A message — received via email, text, chat, or another familiar communications channel — may ask an employee to log in, approve an authentication request, review a document, or change information associated with a business process. AI makes it simpler for threat actors to prepare those messages and do it convincingly.

As Pondurance experts discuss in How AI Is Changing the Cyber Threat Landscape — and What To Do About It, threat actors can use AI to research potential targets and create more believable social engineering attempts. With AI, the quality of the writing is no longer a useful measure of whether a message is safe. The message may be well written either because an actual co-worker sent it or because a threat actor had access to the tools and information needed to make it convincing.

Context matters more than polish

Employees usually have an advantage that threat actors don't have: an understanding of how work normally gets done. Employees know whether a manager typically asks for certain information by email, whether the IT team normally sends login instructions through chat, and whether a vendor usually handles a request in a particular way. 

An unusual context can be one of the strongest signs that something is wrong. Thus, a message deserves more attention when the request feels out of place for the situation. Also, an unexpected sense of urgency can be meaningful, particularly when the employee is being pushed to act before there's time to verify the request.

Employees don't need to analyze every message like a security investigator. Instead, simply taking a brief pause to consider whether the request fits the situation can be enough to interrupt an attack. When a request feels unusual, verification through a separate, trusted communications channel can help. Employees should be encouraged to call a known number or start a new conversation with the alleged sender of the message to halt the threat actor's control over the interaction.

Authentication can be an element of a social engineering attack

Some attacks are convincing because they use legitimate technology. As an example, the Pondurance detection engineering team recently highlighted an incidence of device code phishing, a tactic where a threat actor directs an employee through a legitimate authentication experience and gains access to the network as the user of the device. In device code phishing, an employee may recognize the login page, and the authentication process may work exactly as expected. In this particular incident, the threat actor convinced the employee to enter a code or start a login process that was never requested.

A login request that appears without explanation should stop an employee from taking any further action. Also, when a multifactor authentication prompt arrives but the employee is not actively signing into an account, the employee should stop. Routine authentication should always have context.

Social engineering has moved beyond the traditional phishing email

Threat actors are finding ways to make employees participate more directly in malware delivery. One example is a ClickFix attack, where employees may encounter a fake technical error, security check, or captcha that instructs them to copy a command and run it on their computers. The instructions may resemble ordinary troubleshooting, which makes them difficult to recognize. Pondurance explains the tactic in Cybersecurity 101: A Whole New World of Malware Delivery.

A ClickFix attack works because employees believe they are resolving a problem. But unexpected technical instructions should raise a red flag, particularly when a website requests that an employee open a command line tool or make a security-related change on a device. Organizations should implement policies that allow any employee who is unsure about a potential social engineering attack to easily consult the IT team before continuing.

Reporting can be as important as recognizing the attack

Even experienced employees can be fooled by a convincing message, but the actions taken after the initial incidence can make a significant difference in the outcome of the attack. If an employee realizes that he or she clicked on a suspicious link or approved an unexpected request, reporting the activity quickly gives the security team a chance to investigate while the incident is still developing.

Pondurance's threat intelligence team has emphasized the value of "phishing amnesty," where employees are encouraged to report mistakes quickly without fear that they will be punished for coming forward. Phishing amnesty creates a healthier security culture and gives defenders another source of visibility. After all, the employee who clicked the link may also be the first person who realizes something is wrong that requires the immediate attention of the IT team.

Employees need to trust their instincts

Threat actors can create AI-enabled social engineering messages that look polished enough to pass a quick visual inspection. However, employees bring something deeper to the situation: familiarity with the people, processes, and tools they use every day. That familiarity can help them instinctively recognize when a request seems suspicious.

Cybersecurity Awareness Month is a good opportunity for organizations to make employees aware that a convincing message is not automatically a trusted message. This week, organizations should ask employees to take actions that make it harder for threat actors to succeed, including pausing when a request feels unusual and verifying it through another trusted communications channel before taking action.

A stronger security backstop can help

Employees can't recognize every social engineering attempt, especially as attacks become more convincing due to AI, but there are ways to better the odds of success. Pondurance helps organizations strengthen the layers around their people through managed detection and response, incident response, risk management, and cybersecurity expertise designed to identify suspicious activity before it develops into a larger incident. 

Learn more about Pondurance and explore the many ways organizations can strengthen security to keep employees safe from cyber threats.