Training a machine on ‘bad’ emails
The first AI approach we often see in the wild involves harnessing an extremely large data set with thousands or millions of emails. Once these emails have come through, an AI is trained to look for common patterns in malicious emails. The system then updates its models, rules set, and blacklists based on that data.
This method may be an improvement to traditional rules and signatures, but really it is only automating that flawed, traditional approach – instead of having a human update the rules and signatures, a machine is updating them instead. This is inherently reactive, and will not stop new kinds of attacks.
The industry is beginning to acknowledge the challenges with this approach, and huge amounts of resources are being thrown into minimising its limitations. This includes leveraging a technique called “data augmentation”, which involves taking a malicious email that slipped through and generating many “training samples” using open-source text augmentation libraries to create “similar” emails – so that the machine learns not only the missed phish as ‘bad’, but several others like it – enabling it to detect future attacks that use similar wording.
But spending all this time and effort trying to fix an unsolvable problem is futile. Why try and fix a flawed system rather than change the game altogether?
The rise of ‘fearware’
An attack trend which spells out the limitations of the above approach is fearware. When the global pandemic hit, and governments began enforcing travel bans and imposing stringent restrictions, there was undoubtedly a collective sense of fear and uncertainty.
Cyber criminals were quick to capitalise on this, taking advantage of people’s desire for information to send out topical emails related to Covid-19 containing malware or credential-grabbing links. Attackers purchased over 130,000 new email domains related to Covid-19 to launch these attacks with relative ease.
These emails often spoofed the Centers for Disease Control and Prevention (CDC), or later on, as the economic impact of the pandemic began to take hold, the Small Business Administration (SBA). As the global situation shifted, so did attackers’ tactics.
When we consider the AI approach detailed above, the question becomes: how can you train a model to look out for emails containing ‘Covid-19’, when the term hasn’t even been invented yet?
Covid-19 is the most salient example of this, but the same reasoning follows for every single novel and unexpected news cycle that attackers are leveraging in their phishing emails to evade defences and attracting the recipient’s attention. Moreover, if an email attack is truly targeted to your organisation, it might contain bespoke and tailored news referring to a very specific thing that supervised machine learning systems could never be trained on.
Use cases for AI and ML in cyber security
Use cases for AI and ML in cyber securitySpotting intention
An important approach for future-proofing is to analyse grammar and tone in an email in order to identify intention: asking questions like ‘does this look like an attempt at inducement? Is the sender trying to solicit some sensitive information? Is this extortion?’ By training a system on an extremely large data set collected over a period of time, you can start to understand what, for instance, inducement looks like. This then enables you to easily spot future scenarios of inducement based on a common set of characteristics.
Training a system in this way works because, unlike news cycles and the topics of phishing emails, fundamental patterns in tone and language don’t change over time. An attempt at solicitation is always an attempt at solicitation, and will always bear common characteristics.
For this reason, this approach only plays one small part of a very large engine. It gives an additional indication about the nature of the threat, but is not in itself used to determine anomalous emails.
Detecting the unknown unknowns
The secret weapon of email security is unsupervised machine learning, which starts with extracting and extrapolating thousands of data points from every email. Some of these are taken directly from the email itself, while others are only ascertainable by the above intention-type analysis. Additional insights are also gained from observing emails in the wider context of all available data across email, network and the cloud environment of the organisation.
Only after having a now-significantly larger and more comprehensive set of indicators, with a more complete description of that email, can the data be fed into an impartial machine learning engine to start questioning the data in millions of ways in order to understand if it belongs.
The technology identifies patterns across an entire organisation and gains a continuously evolving sense of ‘self’ as the company grows and changes. It is this innate understanding of what is and isn’t ‘normal’ that allows AI to spot the truly ‘unknown unknowns’ instead of just ‘new variations of known downfalls.’
This type of analysis brings an additional advantage in that it is language and topic agnostic: because it focuses on anomaly detection rather than finding specific patterns that indicate threat, it is effective regardless of whether an organisation typically communicates in English, Spanish, Japanese, or any other language.
By layering both of these approaches, you can understand the intention behind an email and understand whether that email belongs given the context of normal communication. And all of this is done without ever making an assumption or having the expectation that you’ve seen this threat before.
Key cyber security trends to look out for in 2021
Key cyber security trends to look out for in 2021Years in the making
It’s well established now that the legacy approach to email security has failed – and this makes it easy to see why existing recommendation engines are being applied to the cyber security space. On first glance, these solutions may be appealing to a security team, but highly targeted, truly unique spear phishing emails easily skirt these systems. They can’t be relied on to stop email threats on the first encounter, as they have a dependency on known attacks with previously seen topics, domains, and payloads.
An effective, layered AI approach takes years of research and development. There is no single mathematical model to solve the problem of determining malicious emails from benign communication. A layered approach accepts that competing mathematical models each have their own strengths and weaknesses. It autonomously determines the relative weight these models should have and weighs them against one another to produce an overall ‘anomaly score’ given as a percentage, indicating exactly how unusual a particular email is in comparison to the organisation’s wider email traffic flow.
It is time for email security to well and truly drop the assumption that you can look at threats of the past to predict tomorrow’s attacks. An effective AI cyber security system can identify abnormalities with no reliance on historical attacks, enabling it to catch truly unique novel emails on the first encounter – before they land in the inbox.
Written by Dan Fein, director of email security products at Darktrace
Recent Comments