(In)Secure by Design

In 2021, design as a security concern became a top-of-mind issue for application security professionals and developers. To underscore this development, the 2021 OWASP Top 10 added Insecure Design as a new category, emphasizing the importance of threat modeling and secure design patterns. Furthermore, the NIST IR-8397 Guidelines on Minimum Standards for Developer Verification of Software explicitly calls for the use of threat modeling to identify security issues in system design. Cybersecurity standards organizations now finally see the need for security to be an essential consideration in the design and architecture of applications despite concepts like “shift left” and “building security in” being emphasized for nearly two decades. These concepts were made popular by Gary McGraw’s 2006 book, Software Security: Building Security In, which brought these ideas to the forefront of application security.

Despite the significant attention that influential works like McGraw’s brought to application security, many common application security issues highlighted by resources like the OWASP Top 10, SANS Top 20 Software Security Vulnerabilities, and the Common Weakness Enumeration (CWE) database have been persistent and recurrent for many years. Consider injection issues, which have always been in the OWASP Top 10 in one form or another (read more on this in our blog, ‘How to Prevent the OWASP Top 10'. Just when we think we have one type of injection issue solved, some new technology comes along, and we are faced with another.

Why does this happen? One significant reason is the failure to consider the root causes, or security anti-patterns, at the design and architecture level that led to these kinds of recurrent and systemic vulnerabilities. The recurrence of common vulnerabilities such as injection flaws or broken access control is evidence that organizations aren't addressing these anti-patterns early enough in the design process, leading to applications that are inherently insecure by design.

Security Patterns and Anti-Patterns

In application development, there are well-understood secure design principles, or security patterns that, when followed, can guide the design and development of more secure applications. Naturally, for each of these security patterns, there is a corresponding security anti-pattern which can lead to inherent vulnerabilities in a system. These patterns, and their related anti-patterns, are summarized in the table below:

Failure to adhere to these security patterns, and recognize the anti-patterns, has led to several of the issues detailed in every iteration of the OWASP Top 10. For example, the anti-pattern Failure to Establish Trust leads to a recurrent entry in the OWASP Top 10; one that is now at the top – Broken Access Control, as well as another old favorite, Broken Authentication, are now listed as Identification and Authentication Failures.

In addition to the security patterns and anti-patterns mentioned above, we feel there is an additional anti-pattern that is not commonly mentioned and does not have a corresponding pattern – Allowing Data to Become Code; in which software allows untrusted data to be used in an execution context. To understand this anti-pattern, let’s look back to the injection vulnerabilities mentioned in the introduction. Although SQL injection and cross-site scripting (XSS) vulnerabilities are quite different in how they are exploited and their potential impact, they both stem from this same anti-pattern. Untrusted data, such as user input, is accepted by the application and included in an execution context, such as an interpreted SQL statement, or as JavaScript in a rendered web page. Other common injection flaws, such as command injection, LDAP injection, insecure deserialization, JavaScript prototype pollution and even XML external entity (XXE) injection, are a result of this same anti-pattern.

While the traditional mitigation strategy to injection vulnerabilities would be implementing a defense in depth strategy that includes input validation and context-specific output encoding to prevent a specific injection issue, identifying the anti-pattern behind the issues allows us to target the very presence of the vulnerability in a structural manner, during system design.

To better address the anti-pattern of Allowing Data to Become Code, we can inspect the overall design of the system to identify where untrusted data enters the application and flows to components where it may be used by in an execution context. Once those data flows are identified, it is then possible to introduce controls into the security requirements and design, to ensure that the data can only be used safely.

The point at which untrusted data - such as input from a user's browser on the Internet - is accepted by the application is known as a trust boundary. Simply put, a trust boundary is the point at which data flows between components with different privileges. These boundaries may be between an application and the Internet or even within an application if certain processes or resources require different privilege levels. Identifying a system’s trust boundaries and considering the ways those trust boundaries can be leveraged to attack an application is the basis of threat modeling, which has become one of the most widely practiced approaches to identifying potential flaws in the design of a system.

Threat Modeling

Threat modeling helps lay a strong foundation for secure application development by considering the ways in which an application may be attacked and identifying weaknesses in the design and architecture that could leave the application vulnerable to those attacks. The threat modeling process begins with identifying the target application’s trust boundaries, as represented in this Data Flow Diagram illustrating the trust boundaries, noted as red dashed lines, for the major processes of a sample library web application.

Application developers and security professionals then enumerate the threats to each trust boundary by asking the question central to threat modeling: “What can go wrong?” Finally, they propose mitigation techniques to each of these threats.

Armed with a threat model, application architects can identify areas where security patterns or anti-patterns need to be considered and refine the application design and architecture accordingly. Recognizing and remediating the presence of anti-patterns results in systems that are secure by design. It is, however, important to note that while the design may be secure, vulnerabilities may still arise from flaws in implementation, which is independent of the design.

Ultimately, threat modeling leads to better identification of functional security requirements and informs development and testing activities to ensure both security and business functionality are properly implemented. As such, perhaps it is no surprise that both the 2021 OWASP Top 10 and NIST IR-8397 specifically call out threat modeling to support identification of security issues in design.

Conclusion

The recurrence of common vulnerabilities such as injection or broken access control, among others, is evidence that organizations aren't adequately addressing security early enough in the development lifecycle. The increased attention now being drawn to the dangers of insecure design and how to combat it is an explicit call to action for application architects, developers, and security professionals to fully consider security at the design and architecture level. It is clear now that instead of shifting left, organizations need to start left, and bring security into the equation during initial planning and design. By understanding and applying secure design patterns, as well as avoiding anti-patterns, and incorporating security architecture analysis techniques like threat modeling, systemic security issues can be identified and addressed early and help eliminate the perpetual problem of insecure design.

To start implementing security during design phases (and all subsequent phases) in your organization, read our ebook, “Shifting Left: A DevSecOps Field Guide.”