Automating Security Tests – Part 1: Testing for Security Headers

by

Today, performing unit tests has become a standard in many development teams for automatically performing various tests on code (e.g. as a compulsory part of the build process). Especially in agile development, the existence, completeness and quality of such tests is a critical aspect for ensuring that the application is still working after each commited change.

Although unit tests are very often created and used, security relevated tests (aka security unit tests), are still performed very rarely. Such tests can be used to ensure, that code-based security controls (e.g. for authentication, access controls, validation or crypto) are working correctly. Especially for security relevant APIs the existence of proper security unit tests are critical. Also, we can use a unit tests to integrate external security code scanners (e.g. Fortify, Veracode, etc.) and combine it with a proper scan policy for the environment or type of application.

This is not a new concept. Stephen de Vries outlined the need for security unit tests in an OWASP speach titled “Security Testing through Automated Software Tests” in the year 2006.

But we can do even more. We can use the unit testing framework (e.g. JUnit for Java, NUnit for .NET or PHPUnit for PHP) to perform security integration tests as well. These are perhaps not executed with every build, but at least before the code is deployed in production. The following snippied shows an example for a JUnit test based un Apache HTTP Client that performs a test that checks the existence and correct value of a X-Frame-Options response header of a specific Web server:

@Test 
public void XFrameOptionsTest() throws ClientProtocolException, IOException {
   CloseableHttpClient client = HttpClientBuilder.create().build();
   HttpGet request = new HttpGet("www.example.com");
   HttpResponse response = client.execute(request);

   // only for debugging
   System.out.println(request.getRequestLine());
   Header headers[] = response.getAllHeaders();
   for(Header h:headers){
      System.out.println(h.getName() + ": " + h.getValue());
   }

   assertNotNull("X-Frame-Options header used?", response.getFirstHeader("X-Frame-Options"));
		
   String header = response.getFirstHeader("X-Frame-Options").getValue();
		
   assertTrue("X-Frame-Options: DENY?", "DENY".equals(header));
}

We can now execute this test case within our development UI (e.g. Eclipse) or on the command line using mavens surefire plugin. Let’s see how it works with www.google.com (instead of “www.example.com”). Google does set a X-Frame-Options header, but uses the value “SAMEORIGIN” instead of “DENY”. So our second assert should fail:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running secodis.securitytestcases.HTTPHeaderTest
GET http://www.google.com HTTP/1.1
Date: Fri, 20 Jun 2014 14:41:56 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie:  [...]
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
Transfer-Encoding: chunked
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.477 sec <<< FAILURE!
XFrameOptionsTest(secodis.securitytestcases.HTTPHeaderTest)  Time elapsed: 1.378 sec  <<< FAILURE!
java.lang.AssertionError: X-Frame-Options: DENY?
        at org.junit.Assert.fail(Assert.java:88)
[...]

Results :

Failed tests:   XFrameOptionsTest(secodis.securitytestcases.HTTPHeaderTest): X-Frame-Options: DENY?

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Pretty much what we expected. Besides this use case, we could use such integration tests for a large number of additional security checks:

  • Verifying input validation controls
  • Verifying output validation controls< (e.g. an “XSS Detector”)
  • Verifying password policy/complexity checks
  • Verifying session handling and session id lifecycle
  • Verifying access controls
  • Checking for inseure configuration (e.g. error handling)

In the next couple of weeks I will give a few practical examples for such tests and will show how we can integrated various testing frameworks such as Selenium or Watir for that.