IBM Application Security Insider

Enhancing Web Application Security Testing with IBM Security AppScan Glass Box

I have already blogged about AppScan's Glass box (IAST / Runtime Analysis) capabilities a while ago, but I've recently recorded a short demonstration of how to install and run a Glass box scan with IBM Security AppScan Standard.

Here's the Youtube movie (don't forget to watch it in 720p/HD, full screen)

 

Android SQLite Journal Information Disclosure (CVE-2011-3901)

Recently we detected a security vulnerability in Android’s SQLite engine which can be exploited by a non-privileged application in order to disclose sensitive information. It enables a malicious application to eavesdrop on database activities performed by any other application using SQLite, allowing unauthorized access to information such as URL history, messages, and contacts.

The complete advisory can be found here.

Demo of the PoC:

 

We would like to thank the Android Security Team for the efficient and quick way in which they handled this security issue.

Handling Complex Scenarios with AppScan's Custom Parameters

If you are familiar with web technologies, either from assessing them for security, or from developing them, you are probably aware of the many innovative ways developers choose to implement web applications. The days of following the HTTP RFC are long gone - developers do whatever they need to do, in order to make things work.

For example, HTTP Parameters are passed as a part of the URL (e.g. REST), tokens are passed as HTTP headers (e.g. CSRF protection) instead of as cookies or parameters, parameter values are concatenated with weird strings instead of ampersand (&), XML islands inside HTTP request body, JSON, etc.

The task of writing a good, robust and thorough web crawler is almost impossible these days - unless you take into consideration the fact that by the time you finish writing your parser, you'll have to add support for new technologies and new web development paradigms.

What I'm trying to say is that if you're trying to write a good web crawler, you better find a way to enhance it on the fly, and do so in a simple manner, so your users will be able to figure it out.

That's exactly what we've done with AppScan Standard's "Custom Parameters" definition. This capability, which I have already blogged about when I discussed testing of RESTful services with AppScan Standard, allows AppScan Standard users to augment AppScan's HTTP parser capabilities, in almost endless ways. 

The "Custom Parameters" definition, enables an AppScan user to teach AppScan how to extract non-standard parameter structures from HTTP traffic. In addition, once you define such a parameter, you can apply a set of rules specifically for that parameter (or parameter group) - for example, should it be tested? should it be tracked? from the login sequence, or whenever a new value is set by the application, and so forth.

The definition itself is done using regular expressions, which are extremely powerful. You can define where the parameter is located in the HTTP request, if it has a parameter-name, or perhaps only a value, how parameter name/value pairs are concatenated, etc. - it is truly the ultimate way to extend AppScan's HTTP parser to support the most bizarre scenarios.

And why am I suddenly bringing this up?

A few days ago, there was an interesting discussion going on over at Twitter (originally raised by Dan Cornell from the Denim Group), and in a few blogs, about a complex login scenario, which AppScan supposedly required help from Burp proxy. At some point, it was decided that AppScan failed. In reality, all that was needed from AppScan to support that complex login, was a simple "Custom Parameter" configuration.

Once I submitted the sample configuration back to Dan (which was contributed by our own Paul Ionescu - a true AppScan expert), he kindly posted the solution on his blog, and the situation now is that AppScan is currently the only scanner with a validated solution. 

 

Microsoft Anti-XSS Library Bypass (MS12-007)

Introduction:

Microsoft Anti-XSS Library is used to protect applications from Cross-Site Scripting attacks, by providing methods for input sanitization.

Vulnerability:

Microsoft Anti-XSS Library 3.0 and 4.0 are vulnerable to an attack in which an attacker is able to create a specially formed CSS, that after passing through the GetSafeHTML or GetSafeHTMLFragment methods, contains an expression that triggers a JavaScript call in Internet Explorer.

The following ASP.NET code demonstrates the vulnerability:

1. string data = Microsoft.Security.Application.Sanitizer.GetSafeHtml("<html>a<style><!--div{font-family:Foo,Bar\\,'a\\a';font-family:';color:expression(alert(1));y'}--></style><div>b</div></html>");

2. string data = Microsoft.Security.Application.Sanitizer.GetSafeHtmlFragment("<div style=\"font-family:Foo,Bar\\,'a\\a';font-family:';color:expression(alert(1));y'\">aaa</div>");

Explanation:

The string value can be broken down as follows:

div{

font-family:Foo,Bar\,'a\a';

font-family:';color:expression(alert(1));y'

}

A bug in the Anti-XSS library causes the closing apostrophe in the first CSS rule to be dropped. Because of the string not being properly terminated, Internet Explorer now renders this CSS in a different way, which triggers a javascript call:

div{

font-family:Foo,Bar\,'a\a;font-family:';

color:expression(alert(1));

y'

}

Impact:

Every application that relies on either GetSafeHTML or GetSafeHtmlFragment to sanitize user supplied data is vulnerable to XSS.

Remediation:

Microsoft has issued a the Anti-XSS library 4.2 to address this issue.

References:

http://www.securityfocus.com/bid/51291
http://technet.microsoft.com/en-us/security/bulletin/ms12-007

Testing RESTful Services with AppScan Standard

As much as I love SOAP web services (not!), it seems like RESTful web services really caught on and became a de-facto standard these days – you see them everywhere, in the cloud, in AJAX or Web 2.0 applications, mobile applications and so forth.

Unlike SOAP services, RESTful services are lightweight. They are extremely easy to understand and also to develop. Nevertheless, there seem to be a million different definitions as to what they really are, but I think the simplest way to understand them is by using the following four definitions, which I’ve found in this DeveloperWorks article:

1. RESTful services use HTTP methods explicitly
2. RESTful services are stateless
3. RESTful services expose directory structure-like URIs
4. RESTful services transfer XML, JSON or both

Simple right?

As much as RESTful services are simple for humans to understand, they are actually a nightmare for automated web application scanners. Why? Because classic HTTP requests usually include parameters either in the Query or Body part of the request. On the other hand, RESTful services usually pass them as what looks like directories (see rule #3 above). For example – the following HTTP request will return the details for a user named Bob:

GET /data/users/Bob/ HTTP/1.1

Host: www.some.site

Connection: close

 

If this was a standard HTTP request, I would tell you that there’s a good chance you’re looking at a web server that contains 3 directories under its virtual root /data/, /users/, and /Bob/, but that’s not the case. This request, tells the RESTful service to retrieve (GET) the account information for user Bob, which is a part of the /users/ list in our data repository.

When an automated scanner crawls the web application, there’s a good chance that out-of-the-box, it won’t figure out that we’re looking at a RESTful service here, and it will consider these parts of the URL as directories. This means a few things:

• Directory-level tests will be sent to the wrong places – potential false positives
• Parameter-level tests will not be sent to the right places – potential false negatives

IBM's AppScan Standard enables you to train it to cope with RESTful services, using one of two options – Manual or Automatic configuration. Let’s start with the manual option.

Custom Parameters

By default, AppScan automatically recognizes parameters in standard HTTP & HTML formats, but if parameters are in other formats (for example within the Path or within another parameter), you need to define them manually, so that AppScan would be able to recognize, follow and manipulate them during scanning. This is done from the Custom Parameters definition, which you can find under Scan Configuration -> Parameters and Cookies -> Advanced: Customer Parameters.

In order to create a new type of custom parameter definition, you have to click the “+” button, which opens the following screen:

 Let’s see a step by step process of adding a definition that will properly parse and test our “users” parameter in the example above.

1. We’ll start by giving this custom parameter definition the Reference Name RESTful_Path_Parameter
2. In the Pattern field, we’ll enter the regular expression /data/([\d\w\s%]+)/([\d\w\s%]+)/ - This pattern includes two match groups, i.e. /data/group1/group2/, group1 denotes the parameter’s name, and group2 the parameter’s value
3. Since the name of the parameter is the first match group, we will define the Name group index as “1”, and since the value of the parameter is the second match group, we will define the Value group index as “2”. This tells AppScan to extract the name of the parameter from group1, and the value of the parameter from group2. If you are dealing with a Path that only includes a parameter value (i.e. nameless parameters), you can set the Name group index to an empty value, and only mark a single value group
4. Our RESTful service uses Path based parameters, so we’ll set the Location to “Path”. In general, you can set it to either “Body”, “Path”, or “Query”.
5. In our scenario, we’ll leave the Condition Pattern empty. This pattern helps us to limit the behavior of the custom parameter definition, by setting another pattern match on the Location. For example, we could’ve defined the Condition pattern to be: ^/data/, and then our pattern parameter definition would only be relevant for Paths that actually begin with /data/.
6. In addition, in our scenario, we will leave the Response Pattern empty. Just as an FYI - this pattern helps us to teach AppScan how to track the values of our custom parameter in scenarios where the application treats it as a session ID. In such cases, the application might not only embed new values in Paths (e.g. in web links), but also in other places in subsequent responses, such as XML elements, for example: <newSessionID>12345678</newSessionID> - In this case, we would have defined the following Response Pattern:
   <newSessionID>([0-9]+)</newSessionID> - this tells AppScan that even though in the HTTP request, the parameter is called users, it should extract new values from an XML element in subsequent responses, that is called newSessionID. Tricky, complex but nevertheless useful!

That’s it. Once we have our custom parameter definition in place, we can let AppScan crawl and test the application normally. After the Explore phase, you can have a peek in the Data view, and look at the Script Parameters table:

As you can see above, each new RESTful parameter that is extracted and analyzed by AppScan is given a special name in the following format:

__patternParameter__[REFERENCE_NAME]_[NAME_GROUP]__INDEX

In our case, AppScan detected the users parameter with 2 values – Bob and Jane, and the books parameter with two values Bobs Biography and Janes Biography.

The INDEX part of the custom parameter is helpful if the regular expression that we created, caught on the same Path more than once. For example, consider the following Path:

/data/users/Bob/data/phone/areacode/

 our pattern would actually match twice on this Path - the first match (index = 0) would set the parameter name to be users and its value to be Bob, and the second match (index = 1) would set the parameter name to be phone and the its value to be areacode. In such case, the name of the custom parameter would appear as:

__patternParameter__RESTful_Path_Parameter__phone__1

Explore Optimization Module

Mastering AppScan’s custom parameters definition could be a daunting task, but this feature is extremely powerful and will allow you to create complex definitions that could parse non-standard HTTP messages of any type and form. If you are in a hurry, lazy, or simply hate regular expressions, there’s an automated way to detect custom parameters by using AppScan’s Explore Optimization Module, which is available through the Tools->Extensions->Explore Optimization Module (Configure or Run):

This extension runs a smart algorithm that will statistically detect URL rewriting rules, such as those that are heavily used by RESTful web applications to generate its directory structure-like URLs. For example, given enough URLs of the format /data/users/VALUE/... this module will automatically generate a custom parameter definition for you.

How much is “enough URLs”? This depends on the configuration of the module and specifically on its Switch Complexity Limit, which by default is set to 50, meaning that you must have 50 different values for the /users parameter.

If you want this module to automatically kick in during scans, you can enable it by going to: Tools->Extensions->Explore Optimization Module: Configure, and checking the box next to Always run automatically during scans. The module will start working once AppScan has crawled 1,000 URLs. You can increase or decrease this default threshold through the Minimum links to start module configuration. If you suspect that your application is using RESTful services, and the module was disabled when you first scanned it, you can always simply run it by going to:  Tools->Extensions->Explore Optimization Module: Run

After the module ran, AppScan’s scan log will include special messages related to this module, for example:

There you go. All I had to do was to let AppScan crawl the application for a few minutes, then Run the module, and it automatically created a custom parameter definition with the regular expression users/([^/]+)

In general, the more URLs you have, the better this module will behave.

It is also iterative - if you continue scanning the application after it created the first round of definitions, and once it hit the threshold again, or once you clicked on Run, it will refine these rules and create new ones where needed. Simple and elegant, albeit less accurate and powerful than the manual option mentioned earlier. That's it.

This post was a bit long, you probably need a REST now.