“You don’t know what you don’t know, because you can’t see what you can’t see”: Threat detection is reaching a negative inflection point

In this blog post I present a few bold (and quite disturbing) figures about the current state of affairs of malware and its detection. I assert that given the unbearably and unacceptably long time it takes to detect malware (IF at all detected), and the growing cost and complexity of its detection, it’s time to admit that current malware detection technologies have more than exhausted themselves and that it’s time for new detection paradigms to emerge.

The perplexity

Reading the threat and malware reports frequently published by security vendors and security-research labs leaves the reader perplexed. Not one report resemble the others in terms of defining the main threats and the quantitative analysis of the levels and dynamics of the threats.

It seems that each vendor has a “unique angle” on the threat landscape, probably based on its technological solutions and its own security knowledge-base.

There doesn’t seem to be any agreement about the scope and breadth of the threats, save for one conclusion: The threat level is rising and so is its severity.

The harsh reality: Malware true state of affairs

Here are a few bold figures published very recently by prominent security players:

Kindsight (Alcatel-Lucent’s security research lab): The top 5 malware and total of >30% of all malware are MBR and other sub-OS infecting malware

 McAfee: ~80% of all rootkits are MBR (i.e., sub-OS rootkits)

FireEye/Mandiant: The threat detection gap is many months long (median being >8 months!); sandboxes are seriously challenged these days by malware’s advanced evasion techniques

Symantec: Malware Authors Using New Techniques to Evade Automated Threat Analysis Systems.

In simple words: Malware is becoming ever-more sophisticated and persistent. It’s specifically designed to evade both common and advanced detection means.

Using Lockheed Martin’s Cyber Kill Chain topology of malware/cyber attack — we find that the 5th stage, the installation, is the most dangerous stage, and the one least covered by adequate solutions.

Security market dynamics

Endpoint security has long proved to be a weak link in the detection of advanced threats, mainly due to the growing complexity of detecting these malware.

To help overcome this weakness, professional analysis tools were introduced to the front line to deal with zero-day and ever-changing/morphing malware. As we see in these vendors’ own words, these advanced solutions — namely dynamic and static analysis tools, coupled with big-data analysis — are being challenged by the sophistication and fast pace of change and adaptation of these threats.

The somewhat sugarcoated term frequently used is “detection gap”. However, the threat detection state is no less than crisis.

The prerequisites of “successful malware”

Let us take a closer look at what the above figures mean. In order to be “successful”, malware has to have 2 key properties — evasion and persistence:

  1. Evasion: Be undetected while penetrating and initially infecting the victim’s machine
    • This is the phase where the malware employs most of its evasion techniques.
    • Malware authors utilize plethora of evasion techniques designed to evade the numerous methods of malware detection — signature-based and heuristics-based detection, static and dynamic analysis, virtual environments used for sandboxing and so forth.
    • Malware would look for properties of malware analysis framework (being virtualized, size and properties of OS partition & memory and CPU, time of execution, time lapsed since last system reboot, update level of the OS, level of interaction of the “user”, active debuggers and disassemblers/decompilers, “natural” network connectivity to the outside world etc.), and if found the malware would typically not execute, so to prevent detection and analysis.
    • At the same time, the malware would also look for properties of a real human user (user’s interaction with applications — scrolling, mouse clicking, mouse right-clicking etc.) so to assure the target is indeed the desired one.
  2. Persistence:Be undetected DURING and AFTER the machine has been infected
    • During: Malware would install on the hard disk’s boot sectors (i.e., beneath the OS, enjoying superior security privileges than the OS itself), so to boot before the OS boots, gain control over the OS and actively prevent anti-malware solutions from detecting it (e.g., disable or spoof the OS-resident anti-malware software).
    • After: Once the machine has been successfully infected, the malware “shifts” its attention to continue to be undetected. This is typically done utilizing additional evasive techniques (e.g., hiding its data exfilteration activity), hiding itself in parts of the hard disk which are hidden from the OS view (e.g., installing its file system and data storage in unpartitioned disk sectors), and downloading its malicious payload in parts over time (e.g., multi-stage malware).

Threat detection’s negative inflection point

These evasion and persistence techniques are for the most part undetectable by common security means, hence —

you don’t know what you don’t know, because you can’t see what you can’t see

It’s time to admit we’re reaching a negative inflection point, where existing detection technologies have exhausted themselves and are merely adding “more of the same” layers of detection. Without a significant leap forward, which will introduce solutions based on new paradigms, the detection gap will only grow wider.

Some interim conclusions and suggested guiding principles for a new breed of solutions

How detection should be done?

  • Evasion: Given the sophistication and rapid development of evasion techniques, detection of advanced threats must be done not only in sandboxes but in real-life machines, imitating real users to a degree undetectable by “suspicious malware”. This type of entity is a high-interaction honeypot. Unlike existing crude and lab-focused honeypots, this honeypot can be productized to be fully automated, managed remotely and thoroughly hardened to prevent being attacked.
  • Persistence: Given the inherent weaknesses of OS-resident malware detection solutions, and the growing sophistication of the malware’s ability to detect presence of detection technologies, the detection must be completely separated from the attack surface (i.e., the target/victim machine). The detection must be done from below and outside of the target/victim OS. This can only be done using a bare-metal embedded hypervisor, specially designed to detect malware and be immune to malware evasion. Moreover, this detection MUST be done in real-time, to significantly reduce the detection gap, and allow counter measures to be taken before the malware captures it’s desired bounty.

Where detection should be placed?

  • The closest to the targeted users, that is — INSIDE the network and not only at the perimeter or network edge. This way the honeypot will also easily intercept P2P traffic — a grave challenge for existing network-edge sandboxes and network sniffers.

* First published at LinkedIn.

Posted in Uncategorized and tagged , , , , , , , , , , .