Working in infosec means being bombarded with promises of security solutions by vendors with no hard evidence to back up their hype. Illumio knew that their work implementing and managing micro-segmentation for client networks improved their clients’ ability to secure their systems, but without a way to quantify by how much, it was hard to cut through the noise of shady security controls claiming similar benefits.

“It was important for us to differentiate ourselves by providing the numerical benefit of micro-segmentation to organizations. In almost every high-profile breach, the ease of lateral movement has contributed to the devastation. Micro-segmentation as a security capability has been in the back of CISOs’ minds for 5 or 6 years, but now it’s a must-have control.”

— Raghu Nandakumara, Field CTO, Illumio

To prove the value of not just their services, but of micro-segmentation in general as a control, Illumio decided they needed to quantitatively demonstrate how the presence of micro-segmentation would affect the time and effort needed for attackers to access trophy targets. They planned to test its efficacy in an artificial environment where they could methodically control the access between systems and then observe how those restrictions affected unbiased third-party testers simulating an attack on that environment. The testing engagement had two goals:

1. To measure how increased micro-segmentation affected attacker efforts needed to reach trophy targets
2. To measure how micro-segmentation affected attacker efforts when the attack surface increased in size

To establish these unbiased measurements, Illumio brought in Bishop Fox to develop a replicable methodology and then apply it in a testing environment. Illumio had previously engaged with Bishop Fox in a traditional pen testing assessment, and they knew that because of Bishop Fox's ability to approach an environment like a real-world attacker, and their past experience creating frameworks for clients like Google, Bishop Fox was uniquely suited to carry out all aspects of the engagement for Illumio. The assessment team could design the testing framework, use red team tactics to implement the attack scenario, and then advise the Illumio team afterwards about the quantifiable effectiveness of micro-segmentation.

“While theoretically we could have done this testing in house, it was important for objectivity that we reach out to established red team experts to build a transparent testing methodology, ensure that it was sound, and advise us honestly throughout to learn exactly how and where Illumio provides value.”

— Raghu Nandakumara, Field CTO, Illumio

Building a First-of-its-kind Methodology

Before deciding on the specifics of the tests, Illumio wanted Bishop Fox to build the test scenario on top of a solid methodological framework that would represent a true attack on the micro-segmentation in their test environment.

MITRE ATT&CK is a well-respected security framework that was developed as a way for organizations to better understand the motivations and techniques used by attackers on their environments. Normally, the MITRE ATT&CK framework is used by an organization as they consider how attackers could move laterally or escalate privileges in their current system, and then using that knowledge to choose which security controls could best secure their specific environment.

In this Illumio experiment however, Bishop Fox knew ahead of time that micro-segmentation would be the security control in use, so the team realized they could use the structure of the MITRE ATT&CK framework backwards to reverse-engineer how a real-world attacker would most likely try to bypass these controls. This meant that their testing would directly assess micro-segmentation’s efficacy as a security control.

Testing Micro-Segmentation

The team had Illumio set up three use cases with incrementally greater restrictions on what each hosts could connect to. During each test, the team would work to identify any exposed ports within the 100 workloads (hosts) in each environment, and then use those exposed endpoints to pivot further into the network until they accessed the trophy target (e.g., the crown_jewels database). As the segmentation increased in each test, the team had fewer initial ports to access, and fewer viable connections that they could use to continue their attack chains. After each test, Illumio randomized the possible connections within the environment, so the Bishop Fox team would know their objectives but not which pathways would lead them to the trophy targets.

To ensure that the total time that it took them to access the trophy target was not inflated by their first time interacting with the artificial environment setup, the assessment team performed a control test twice against a flat network with no micro-segmentation, reaching the trophy target quickly both times.

“Lack of segmentation could be written up as an issue on 90% of the tests we do – we commonly see organizations with networks comparable to the control test here. Organizations know this is a problem, but attempting to implement segmentation at scale can be daunting on your own.”

— MJ Keith, Principal, Bishop Fox

Once they had established a baseline time for accessing the trophy with no constraints in the control test, they performed attacks against the three increasingly segmented environments.

After those three tests, the team tested Use Case Two two more times within larger and larger artificial attack surfaces (with 500 and 1,000 hosts) to measure how micro-segmentation would affect attacker effort at scale.

Reporting the Results

The three use cases each implemented an increased level of segmentation in the network’s infrastructure, which each limited the potential pathways to the crown_jewels. The team found that as micro-segmentation increased, so did their times in reaching the trophy targets:

The findings showed that implementing micro-segmentation increased the difficulty of an attacker to reach their target up to 22x, depending on the degree of micro-segmentation.

In addition, as the number of workloads (hosts) in the test environment increased from 100 to 500 to 1,000, the time required by Bishop Fox’s red team to reach the trophy target doubled. Extrapolating from those results, Bishop Fox’s team found a strong correlation between increased micro-segmentation, increased attack surface size, and increased attacker effort required.

“One of the key takeaways from this report is that as the size of a protected estate increases, the attacker’s job gets measurably more difficult, even as the granularity of micro-segmentation policy is kept constant.”

— Ronan Kervella, Senior Security Consultant, Bishop Fox

Anecdotally, Illumio had known that their micro-segmentation services improved the security of their clients’ internal environments, but by proving it objectively and sharing the results publicly, they have increased awareness and confidence in micro-segmentation as a security control within their teams and to prospective users.

Benchmarking Security

It’s impossible to know an attacker’s strategies and motivations, but through the unique methodology developed by Bishop Fox for this red team scenario, Illumio received insight about how their micro-segmentation capability affected the time and effort that attackers would need to discover and move across a deployed network.

“In some ways, these use cases represented best case scenarios for attackers – a finite environment with microsegmentation as the known security control and no alarms to avoid setting off. However, as the attack surface expanded, we were forced to change tactics to give us a chance to get to the crown jewels in a reasonable amount of time, even with those benefits.”

— MJ Keith, Principal, Bishop Fox

Innovation is a crucial aspect of infosec, but the new ground constantly being broken also means that it’s hard to tell what is actually happening and which products have value. By benchmarking your security products, you can objectively measure the benefits that customers will receive compared to no security in place, or whatever the last popular, standard, unmeasurable product was. Third-party experts like Bishop Fox can aid in that goal by doing what they do best – tackling new methodological challenges and testing environments with their experience and real-world techniques to better understand the strengths of your services.

About Illumio

Illumio enables organizations to realize a future without high-profile breaches by preventing the lateral movement of attackers across any organization. Founded on the principle of least privilege in 2013, Illumio provides visibility and segmentation for endpoints, data centers or clouds. The world’s leading organizations, including Morgan Stanley, BNP Paribas, Salesforce, and Oracle NetSuite, trust Illumio to reduce cyber risk.