In 2021, design as a security concern became a top-of-mind issue for application security professionals and developers. To underscore this development, the 2021 OWASP Top 10 added Insecure Design as a new category, emphasizing the importance of threat modeling and secure design patterns. Furthermore, the NIST IR-8397 Guidelines on Minimum Standards for Developer Verification of Software explicitly calls for the use of threat modeling to identify security issues in system design. Cybersecurity standards organizations now finally see the need for security to be an essential consideration in the design and architecture of applications despite concepts like “shift left” and “building security in” being emphasized for nearly two decades. These concepts were made popular by Gary McGraw’s 2006 book, Software Security: Building Security In, which brought these ideas to the forefront of application security.
Despite the significant attention that influential works like McGraw’s brought to application security, many common application security issues highlighted by resources like the OWASP Top 10, SANS Top 20 Software Security Vulnerabilities, and the Common Weakness Enumeration (CWE) database have been persistent and recurrent for many years. Consider injection issues, which have always been in the OWASP Top 10 in one form or another (read more on this in our blog, ‘How to Prevent the OWASP Top 10'. Just when we think we have one type of injection issue solved, some new technology comes along, and we are faced with another.
Why does this happen? One significant reason is the failure to consider the root causes, or security anti-patterns, at the design and architecture level that led to these kinds of recurrent and systemic vulnerabilities. The recurrence of common vulnerabilities such as injection flaws or broken access control is evidence that organizations aren't addressing these anti-patterns early enough in the design process, leading to applications that are inherently insecure by design.
Security Patterns and Anti-Patterns
In application development, there are well-understood secure design principles, or security patterns that, when followed, can guide the design and development of more secure applications. Naturally, for each of these security patterns, there is a corresponding security anti-pattern which can lead to inherent vulnerabilities in a system. These patterns, and their related anti-patterns, are summarized in the table below:
Pattern | Anti-Pattern |
Defense in Depth - A system leverages several security controls at different layers working in different ways such that if an attack gets past one defense, it is blocked by another. | Single Point of Failure - Traditionally, this refers to a single malfunction that can cause the entire system to fail. In the context of application security, this represents a single security control that, if bypassed, would give an attacker access to the system. |
Fail Safe Defaults - When a system experiences failure, it should do so in a way that prevents damage to the system. A common fail-safe default is explicit deny, where a subject is denied access to an object by default, unless it is explicitly granted access to it. | Failing Open - When a system experiences failure, it falls back to a less secure state, usually in an effort to maintain system functionality even in cases of system error. |
Least Privilege - A subject (user, program, script, etc.) only has the permissions necessary to perform its current task. If the subject must complete tasks that require higher permissions, only those tasks, and no other tasks assigned to the subject, are performed with higher permissions. | Execution with Unnecessary Privileges - Subjects have more permissions than required to perform tasks. |
Separation of Duties/Privilege - A given, and often sensitive, action requires the participation of more than one party to run. For example, two operators are needed to launch a nuclear missile. | Reliance on a Single Factor - Sensitive actions only depend on a single condition being met or approval from a single party. |
Economy of Mechanism - A system only uses services and protocols that are essential because unnecessary functionality or services increase the system’s attack surface. | Unnecessary Complexity - A system uses mechanisms that are overly complex or offer non-essential functionality, which leads to improper implementation or lacking security checks. |
Complete Mediation - Every time a subject requests access to an object, its authorization is verified. This ensures that authorization controls cannot be bypassed. | Failure to Establish Trust - A system does not sufficiently authenticate and authorize the users and components accessing its resources and functionality. |
Open Design - A system shouldn’t be secured by the fact that its design is hidden or secret. For example, the algorithms for common encryption mechanisms are publicly known, as their security comes from the secrecy of the key, not the algorithm itself. | Security by Obscurity - A system is obfuscated such that the difficulty of understanding it (incorrectly) serves as protection against attacks. |
Least Common Mechanism - In order to prevent the unnecessary sharing of information within a system, processes should not share a common mechanism, but rather have unique, separate mechanisms. | Insufficient Compartmentalization - Functionalities and processes with different permissions are combined in such a way that the functionality of lower-privileged users may impact higher-privileged users. |
Psychological Acceptability - Security controls in a system must not be so complex that users view them as barriers to productivity and, therefore, try to work around them. The system must be easy to use, and security should not be an added burden to the user. | Insufficient Psychological Acceptability - A system’s security controls are too inconvenient to users, who then work to bypass the controls in order to avoid the burden presented by the security mechanism. |
Failure to adhere to these security patterns, and recognize the anti-patterns, has led to several of the issues detailed in every iteration of the OWASP Top 10. For example, the anti-pattern Failure to Establish Trust leads to a recurrent entry in the OWASP Top 10; one that is now at the top – Broken Access Control, as well as another old favorite, Broken Authentication, are now listed as Identification and Authentication Failures.
In addition to the security patterns and anti-patterns mentioned above, we feel there is an additional anti-pattern that is not commonly mentioned and does not have a corresponding pattern – Allowing Data to Become Code; in which software allows untrusted data to be used in an execution context. To understand this anti-pattern, let’s look back to the injection vulnerabilities mentioned in the introduction. Although SQL injection and cross-site scripting (XSS) vulnerabilities are quite different in how they are exploited and their potential impact, they both stem from this same anti-pattern. Untrusted data, such as user input, is accepted by the application and included in an execution context, such as an interpreted SQL statement, or as JavaScript in a rendered web page. Other common injection flaws, such as command injection, LDAP injection, insecure deserialization, JavaScript prototype pollution and even XML external entity (XXE) injection, are a result of this same anti-pattern.
While the traditional mitigation strategy to injection vulnerabilities would be implementing a defense in depth strategy that includes input validation and context-specific output encoding to prevent a specific injection issue, identifying the anti-pattern behind the issues allows us to target the very presence of the vulnerability in a structural manner, during system design.
To better address the anti-pattern of Allowing Data to Become Code, we can inspect the overall design of the system to identify where untrusted data enters the application and flows to components where it may be used by in an execution context. Once those data flows are identified, it is then possible to introduce controls into the security requirements and design, to ensure that the data can only be used safely.
The point at which untrusted data - such as input from a user's browser on the Internet - is accepted by the application is known as a trust boundary. Simply put, a trust boundary is the point at which data flows between components with different privileges. These boundaries may be between an application and the Internet or even within an application if certain processes or resources require different privilege levels. Identifying a system’s trust boundaries and considering the ways those trust boundaries can be leveraged to attack an application is the basis of threat modeling, which has become one of the most widely practiced approaches to identifying potential flaws in the design of a system.
Threat Modeling
Threat modeling helps lay a strong foundation for secure application development by considering the ways in which an application may be attacked and identifying weaknesses in the design and architecture that could leave the application vulnerable to those attacks. The threat modeling process begins with identifying the target application’s trust boundaries, as represented in this Data Flow Diagram illustrating the trust boundaries, noted as red dashed lines, for the major processes of a sample library web application.
Application developers and security professionals then enumerate the threats to each trust boundary by asking the question central to threat modeling: “What can go wrong?” Finally, they propose mitigation techniques to each of these threats.
Armed with a threat model, application architects can identify areas where security patterns or anti-patterns need to be considered and refine the application design and architecture accordingly. Recognizing and remediating the presence of anti-patterns results in systems that are secure by design. It is, however, important to note that while the design may be secure, vulnerabilities may still arise from flaws in implementation, which is independent of the design.
Ultimately, threat modeling leads to better identification of functional security requirements and informs development and testing activities to ensure both security and business functionality are properly implemented. As such, perhaps it is no surprise that both the 2021 OWASP Top 10 and NIST IR-8397 specifically call out threat modeling to support identification of security issues in design.
Conclusion
The recurrence of common vulnerabilities such as injection or broken access control, among others, is evidence that organizations aren't adequately addressing security early enough in the development lifecycle. The increased attention now being drawn to the dangers of insecure design and how to combat it is an explicit call to action for application architects, developers, and security professionals to fully consider security at the design and architecture level. It is clear now that instead of shifting left, organizations need to start left, and bring security into the equation during initial planning and design. By understanding and applying secure design patterns, as well as avoiding anti-patterns, and incorporating security architecture analysis techniques like threat modeling, systemic security issues can be identified and addressed early and help eliminate the perpetual problem of insecure design.
To start implementing security during design phases (and all subsequent phases) in your organization, read our ebook, “Shifting Left: A DevSecOps Field Guide.”
Subscribe to Bishop Fox's Security Blog
Be first to learn about latest tools, advisories, and findings.
Thank You! You have been subscribed.
