Automating DAST Scans with Jenkins, Arachni & ThreadFix

I’m often asked how security tests can be automated with non-commercial tools, e.g. triggered by a Jenkins build. Therefore I decided to write this post, to give you a bit of understanding which tools you can use and what you have to do in order to accomplish this goal.

To not over complicate this, I will only focus on tools that finds vulnerabilities in custom code and application config, such as SQL Injection or Cross-site Scripting (XSS).

As with commercial tools we basically have three types of security test tools that we need to distinguish here: static code scanners (SAST), dynamic code scanners (IAST) as well as dynamic web scanners (DAST). Especially for latter, a couple of good and free tools exist that we can use here. The most popular ones are at the moment most likely OWASP ZAP and Arachni. I worked with both tools and personally find Arachni to be the better suitable tool, especially for automated scans. I will therefore focus only Arachni here. Although my examples are based on integrating Arachni into Jenkins, I tried to only use functionality that should be available in any other CI as well.

Architecture

The following diagram visualizes the components and its interactions described in this post. We have a Jenkins CI, a git repository (could be SVN or any other code repository as well), a tomcat as well as the both tools this post is about: Arachni for scanning and ThreadFix as a database where the results are stored and analyzed.arachni arch
Of course you may also be implement Arachni differently or use other components.

Preparations

First we need a vulnerable demo app so that we can scan with Archni and see whether its working or not. I’ve created a rather simple Java-based WebApp that basically has one HTTP form with reflected Cross-site Scriptings (XSS) in each form field that can be exploited via HTTP POST parameters “age” and “name”:

insecurewebapp1

The corresponding HTTP request looks like this:

Then we of course need a Jenkins installation set-up, that build our web app and deploys it to a app server. In this case I created a job called “insecure-webapp” for our demo app and used Jenkins Tomcat Plugin for its automatic deployment.

Installing Arachni

The installation of Arachni is pretty simple. You just need pick the right version here download it on the System where your Jenkins (or other CI) is running and extract it there. That’s it.

Integrating Arachni into Jenkins

Arachni provides a couple of different interfaces that we can use for automation. Besides a Web GUI there is also a command line interface (CLI) as well as REST and an RPC service that we can trigger. Although one of the latter two seems best suitable for automation, I find the CLI to be the most comprehensive one that is also very easy to integrate. The CLI can be very simply integrated by a shell as a post build step (no Arachni Jenkins plugin exist anyway):

jenkins-arachni-start-1-768x162
In this case I just told Arachni to crawl the provided URL but only scan for XSS vulnerabilities. This configuration is good starting point for using Arachni. It is of course not a sufficient configuration for identifying all common Web vulnerabilities, especially not for an enterprise app! The CLI provides a lot of options, that you would perhaps need, especially when you want to scan a large application, selecting and configuring test cases, including/excluding certain URLs and providing authentication credentials.

Running Jenkins with Arachni

The next time the build is executed, Jenkins automatically grabs the source code from the repository, builds it, deploys it on the Tomcat and scans it with Arachni as we can see it on the following console output (stripped for demonstration purposes):

As we can see above, Arachni actually finds XSS vulnerabilities in both vulnerable HTTP parameters (“name” and “age”) and does this within every build! However, the build still succeeds, since we do not do anything with the Arachni results.

Breaking the Build

If we want to flag a build as unstable if Arachni finds a security problem, we need to do a little bit of extra work. As we can see in the console output, in case Arachni didn’t find anything, it outputs “0 issues were detected”. We can now easily parse the output for this string with Jenkins Text Finder Plugin that is executed as another post build action

If this string is not present, we assume that Arachni found something and tell the Finder Plugin to mark build as unstable. The result is the following output for a positive security finding:

.. resulting in an unstable build:

jenkins-build-failed

Sending Findings to ThreadFix

Regardless of whether you want to have your builds automatically failed when certain vulnerabilities has been found or you just want to monitor existing findings in your applications, ThreadFix is a great tool for that.

ThreadFix is a web-based tool for collecting findings from different tools such as Arachni. There is a Jenkins plugin available that can be integrated via an additional post build action step very easily so that findings are automatically send to ThreadFix where thex can be monitored and assessed via an Web interface.

threadfix screenshot

To be able to parse Arachni scan output, you must use the Arachni report command to convert the .asf files to .xml files via an additional build step though. I use an additional conditional post build step for this that checks if an Arachni report file exists and runs shell command

arachni_reporter ${BUILD_TAG}.afr –reporter=xml:outfile=${BUILD_TAG}.xml

to get the file format that we can upload via ThreadFix Jenkins plugin into our ThreadFix vulnerability database as shown in the screenshot above.

There is a community edition of ThreadFix that lacks of some enterprise features (such as ACL based on users / teams, SSOs, etc.) but can be uses without costs, even in an commercial environment.

Advanced Configuration

We can of course scan much more vulnerabilities besides just XSS. And we should. When you do not specific test cases Arachni will automatically scan for everything, including platform fingerprinting and SSL checks. Be careful with that as well, because this will most likely produce a lot of false positive. Instead, try to find out which test cases are useful for the tested technology stack (e.g. no SQL injection tests when you are shure that you have a Mongo DB). Start with a simple set-up and include more step by step.

Also, Arachni provides a number of ways to login into applications to perform deep scans. Do not scan authenticated sessions in production though since this can have a lot of problematic side effects.

Limitations & Other Tools

As mentioned, even with a highly customized Arachni configuration, the approach described here is only supposed to be a cheap and efficient approach for identifying low hanging fruits within the custom code and application config of an Web application, nothing more. If you want to cover more vulnerabilities (or identify them ealier in your SDLC) with free tools, you should also concider using dependency checkers such as OWASP Dependency Checker (for Java only) as well as static code scanners such as Findbug Security Plugin or similar tools.

Especially in this tool categories, both scan quality and integration capabilities of free tools are still very limited at the moment and far behind commercial tools such as Contrast IAST or Checkmarx SAST.

About Matthias Rohr

Matthias Rohr, CISSP, CCSLP, CISM, CCSK, is Founder and CEO of Secodis GmbH. Matthias began working in this field in 2004 and is since a frequent speaker on international conferences, book author and an active OWASP constributor and recently published his first book. He lives and works in Hamburg / Germany.
This entry was posted in DAST, Java, Security Test Automation, Uncategorized and tagged , , , . Bookmark the permalink.

4 Responses to Automating DAST Scans with Jenkins, Arachni & ThreadFix

  1. Beate Hillebrand says:

    We want to automate Pen-testing with Jenkins and consider using Arachni or OWASP ZAP. Why do you think, Arachni is the better tool, especially for automated scans?

    • I wouldn’t say that Arachni is better then OWASP ZAP. I worked with both tools (as well as w3af and of course Burp that others prefer) but do in fact prefer Arachni for automation. One reason is that I figured that Arachni is really easy to automate. Perhaps this is because it is mainly build for this purpose and provides a good CLI as well as a number of scanning services such as RPC whereas OWASP ZAP main interface is the UI. All these tools are great but can be more or less suitable for scanning a certain websites that requires complex configurations or specific test (OWASP ZAP may be more comprehensive here). In this case the decision is easy. I would not combine different DAST scanners though, at least not if you plan to scan complex sites, because configuration of DAST scanners can get really tricky. At least if you are not putting a lot of effort and expertise in it, DAST scanning will always be a baseline scan that can not replace a manual pentest.

  2. rany says:

    Hello,

    Nice blog on security automation , I have one question , how can we reduce false positives during security automation tests ? . In short, filtering false positives before generating final scan result or parsing Arachini Scanner scan output before storing into ThreadFix database . Is there any way for this ?

    • Hi Rany,

      Sorry for my late response. This is in fact a major concern in respect of security test automation and not a single but many answers to this:

      • Run tests early and often: You will experience most false positives in the first scans
      • Adapt scan policy: Customize scanning rules to your own technology stack (e.g. specify exceptions or your own cleansing functions)
      • Focus on simple tests: Scans that are performed continuously (e.g. within CI/CD) should be simple with a high true positive rate. Custom security unit or integration tests (e.g. that verify permissions, tokens, headers or something like this) are good example for this. Full DAST or SAST scans could be executed periodically by an analyst but not within the pipeline if they throw to many FPs
      • Select suitable tools: There is no best security tool on the market only a best suitable tool. We at Secodis do a lot of tool evaluations and find that some tools work good with some technology stacks while others provide reliable results for other stacks. Before you use a tool make sure that it is suitable for your stack (the most expensive tool on the market is not necessarily the best suitable tool for your!)

      Hope this helps a bit!
      Matthias

Leave a Reply

Your email address will not be published. Required fields are marked *