Choosing the Right Modern Application Security Tools

The latest trend toward Development, Security, and Operations (DevSecOps) is intended to ensure that security is embedded into every phase of the Software Development Life Cycle (SDLC). The goal is to make security a shared responsibility among everyone in the development organization (developers, infrastructure teams, and security). This approach requires many tools that need to integrate within the development process to build security in without slowing development down. While some delays are to be expected as a necessary cost for greater overall security, the product timeline can’t come to a screeching halt.

One of the greatest sources of pain in the application security (AppSec) industry is trying to understand the plethora of tools and technology that are available for discovering flaws in applications. It seems like every week or so, a large AppSec vendor or startup is announcing the latest and greatest “innovative” tool to solve all of your AppSec problems which, by the way, will perfectly integrate into your Continuous Integration/Continuous Delivery/Deployment (CI/CD) pipeline automating all of the security things our developers have ever dreamed about. That last sentence is said with great sarcasm, but the reality is, no single AppSec tool (or vendor for that matter) will solve all of your testing challenges.

AppSec vendors offer many different approaches and technology that may not always fit into the way your organization develops applications.

AppSec tools and related technology are always evolving, so this post will provide a vendor agnostic approach to the advantages and disadvantages of modern AppSec tools. I am a firm believer in a “keep it simple” approach, combined with using one or more of these tools, for the way your organization develops applications. AppSec is not a one size fits all approach when it comes to tooling. And no matter what buzzword or marketing propaganda a vendor might throw at you like AI, machine learning, blockchain powered, or “we’re in the Gartner magic quadrant for the 100th time…again” at the end of the day, it is about choosing what works best for your organization. So let’s get started with the grand daddy of all AppSec tools, Static Application Security Testing, or SAST for short.

STATIC APPLICATION SECURITY TESTING (SAST)

SAST is the most mature technology out of the modern tools we’ll be discussing in this blog post. SAST (also known as static analysis) was one of the first AppSec tools in the market that focused on scanning code to find flaws in source code, byte code, or through the binary itself. Some SAST tools will scan the uncompiled source code while others will scan the compiled binary where source code is not required. This is known as binary analysis. Some SAST tools are installed on premises, within a developer IDE, and others utilize a Software as a Service (SaaS) model using the “cloud.” There are many different types of open source and commercial SAST products in the industry today, and it is the one tool with the largest choice in vendors.

Advantages

• Early Detection: Because SAST tools are as far “left” (i.e. focused on finding coding flaws early in the development cycle) that you can go in AppSec testing, SAST can find more common code-related flaws as early as possible.

• Scale with Developers: SAST tooling can more easily scale across multiple developers and applications because of how close this tool is to the development process.

• Build Cycle Integration: SAST tools are typically most efficient in mature DevSecOps environments where SAST tools can be tightly integrated into the build cycle.

Disadvantages

• Language Support: SAST tools can vary greatly on the types and number of programming language frameworks they support. Often vendors require lead-time to support these new languages and frameworks. Language support and associated timelines are typically built into their product roadmaps which can change frequently.

• Scan Duration: Depending on the size of the code base, complexity of third-party libraries, and the binary itself some scans can take a long time to complete. In recent years, scan times seem to be improving as many vendors continue to evolve this technology. However, it’s important to note that scan times can vary greatly across different SAST products and vendors.

• False Positives: Some SAST tools have higher false positive rates than others. That is why SAST results need to be manually reviewed (against source code) by someone experienced in identifying security and coding flaws. For example, most SAST scanners will report “hard coded credentials” because it flags a particular keyword in the code that it believes is a security issue. Without manual validation, this can lead to many false positives with this flaw in particular.

• Missing Flaws: SAST scans cannot detect flaws like Insecure Direct Object Reference (IDOR) or business logic flaws. Flaws like these can only be identified through penetration testing or manual source code review.

DYNAMIC APPLICATION SECURITY TESTING (DAST)

DAST technology crawls and analyzes an application in its runtime state and can only identify flaws and configuration issues that its scan engine knows about. DAST scanners typically test exposed HTML interfaces, but more evolved DAST scanners can also test REST APIs. One source of confusion is that many people think that a dynamic scan is an “automated” penetration test. This is simply not true. A dynamic scan is considered a vulnerability assessment as it tests for flaws the scanner knows about. Dynamic scans do not conduct exploitation of flaws like a skilled penetration tester does, therefore, is not the same as a penetration test and won’t catch zero-days or emerging threats. There is also a difference between a network vulnerability scanner and an application vulnerability scanner. Some vendors will claim to do both, but rarely will a vendor have a product good at both.

Advantages

• Ease of Use: Most scanners these days are “point and shoot,” meaning that you give the scanner a URL and it does its thing.

• Simpler Validation: Decent DAST tools will give you the actual URL and very specific details of flaws identified. This can make manual validation much easier.

• New Capabilities: Many modern DAST tools can scan REST APIs when years ago, DAST tools didn’t support scanning API. Manual testing has traditionally been used to test APIs.

• Improved Integration: DAST tools are becoming easier to integrate into functional testing and continuous scanning in a pre-prod or production environment. Many DAST vendors now offer APIs and direct integration into bug and vulnerability tracking systems.

Disadvantages

• Performance Impact: DAST scanners send tons of traffic to an application which can make it unstable and, in some cases, cause the application to crash. Also, it is common to have things like contact forms auto-submitted (spamming your help desk) and the application’s database populated with test strings and other junk data. Therefore, DAST scans should be either finely tuned or ran in pre-production environments.

• Scan Duration: Depending on the complexity of the application, some DAST scans can run for days. Some DAST scanners can be paused; others will just run until the scan is complete.

• Framework Challenges: Single page applications or applications that use very modern frameworks can cause problems for some DAST scanners.

• Vendor Variability: Scan speed and even results can vary greatly between vendors. Make sure you do your homework and/or have a bakeoff to determine who is the best fit for your needs.

• Configuration challenges: Configuring authenticated DAST scans (using credentials or certificates) can be challenging and may require a lot of troubleshooting and custom coding.

• Missing Flaws: Like SAST, DAST tools cannot find flaws like IDOR, business logic or privilege escalation flaws. These flaws can only be identified through penetration testing or manual source code review.

SOFTWARE COMPOSITION ANALYSIS (SCA)

SCA tools have really gotten popular in the last several years due to the growing risk open source presents. This technology is focused on open source risk and identifying open source vulnerabilities in source code, containers, and registries. Most SCA tools are also able to check for open source licensing compliance and attribution requirements.

Advantages

• Catch Open Source Flaws: SCA can identify open source vulnerabilities that are often overlooked by developers. It’s very common for developers to just grab some random open source library, use it in their application, and not realize that the library has known security vulnerabilities, unknowingly embedding these known vulnerabilities into their application. SCA tools solve this problem.

• Ease of Integration: Most SCA tools can directly integrate into CI/CD pipelines.

Disadvantages

• Vulnerability Delays: Open source vulnerabilities are always being released so SCA vendors have to maintain a fairly large vulnerability database. Having said that, updates to these databases can take time, so you’re never really getting a real-time view into your open source risk.

• Missing Flaws: SCA tools cannot identify unknown flaws in open-source components. This can only be done through penetration testing or manual source code review.

INTERACTIVE APPLICATION SECURITY TESTING (IAST)

IAST is a newer technology that tests for flaws while the app is being interacted with from a user perspective. Typical IAST implementations are a mix of SAST and DAST deployed with agents or sensors running within the application. This technology has not gained a lot of traction (yet) since its still so new in the industry.

Advantages

• Instant Gratification: Results are typically in real-time while the application is being used. Many organizations are finding IAST works best during functional QA testing.

• Functional Testing: API testing can greatly benefit from IAST because of the functional nature of how APIs are designed.

• Ease of Integration: Integrates well into a CI/CD pipeline

Disadvantages

• Limited Scope: IAST does not test for flaws in the entire application. IAST will only test the parts of the application being used.

• Language Support: IAST technology is still immature and language support is still limited.

• Performance Impact: Some IAST technology is known to cause performance issues with the application.

RUNTIME APPLICATION SELF PROTECTION (RASP)

RASP is a newer technology that is integrated into an application to analyze user traffic and behavioral patterns to block and prevent attacks. RASP is deployed on production servers and works more like a traditional web application firewall (WAF) but instead of regular expression matching, RASP technology uses AI and machine learning.

Advantages

• Stack on Security: RASP can add an extra layer of security for applications that are already coded securely.

• Zero-Day Protection*: Because most RASP vendors are using AI and machine learning, in theory, RASP can help protect against zero-day attacks.

• Production Feedback: RASP can provide a good feedback loop to developers to see how their applications are being attacked in production.

• Broad Integration: RASP solutions have evolved to support cloud infrastructure deployments and more vendors support integrating RASP into CI/CD pipelines.

Disadvantages

• Performance Impact: Because RASP runs on an application server in production there is the potential for performance issues.

• False Sense of Security: RASP is not a security testing tool, rather it should be used as a “defense in depth” solution. Meaning, RASP should be deployed after the application is coded securely. Note that RASP can be prone to leaving an organization with a false sense of security if relied on to secure apps that have coding flaws.

• Solution Immaturity: RASP is still not widely adopted because the technology is still considered immature.

WHERE DOES APPLICATION PENETRATION TESTING (APT) FIT?

You may be asking yourself, "all of these automated tools are great, but what about penetration testing?" You can look at penetration testing as another tool, just like SAST and DAST, that should be part of your overall testing strategy. In fact, many in the industry would agree that it’s one of the most important types of testing because penetration testing finds flaws that automated tools cannot. If you’re not conducting at least some penetration testing on your applications, you may be missing very serious flaws like IDOR, business logic, or privilege escalation. These are frequently found flaws that are the source of many data breaches and leaks you hear about in the news almost daily. The key is to find the right balance between automation and manual testing.

CONCLUSIONS

There are several modern automated tools that are available for DevSecOps and Application Security teams to choose from. Note that one tool is not a replacement for another as each technology discussed in this post has its own advantages and disadvantages. Nor can an organization rely on a single technology or testing tool to find all flaws in an application. It should also be said that no tool is 100% accurate. You still need to manually review your code and validate any result that comes from an automated tool. The key is to understand how these technologies can be leveraged to support the way your organization develops applications and choose what is best for your development teams.

Interested in this topic? WatchTom Eston's on-demand webinar “DevSecOps & Application Penetration Testing: Defying the Myth”