WAF versus DPI Firewall

This is a question, I've frequently been asked in recent years and in the last month, on one of the internal mailing lists, in my old company, the following question was posted –

In simple terms, what tasks is a Web Application Firewall (WAF) able to do that a Deep Inspection Firewall can't and why ?

by one of my colleagues.

Many of you may be surprised (I know I was initially) but this question still comes up an awful lot. Having answered the email (as a warning, I went into a lot of detail and plugged the awesome Security Onion), I was requested to write a technical blog on the subject, but as I left the company soon after, the blog was never published. Therefore, to save me answering the question again, I thought I’d publish it so I can just reference the link in future J

Below is the answer that I gave, along with a little history -

In my humble option, they're two very different and distinct technologies.A WAF works solely at layer 7. The configuration should be closely matched to the profile of the web application, i.e. the AppFw policy will resemble the application either through learning or specific signatures, e.g. don't apply PHP and Apache signatures to an IIS server running .net framework.

A deep packet inspection (DPI) firewall is a bit of a marketing term that the traditional firewall vendors (and analysts) have been pushing for a few years and it marks how they've transformed from firstly basic packet filters to then stateful packet filters (as they began to become session aware, e.g. passive ftp sessions) and then to dpi. In a previous life, I ran a firewall team, and we felt DPI was a very limited feature on our firewalls (a bolt-on almost to appease customers) with only a few protocols properly understood and even those were limited with several false-positives. As a result, we turned it off L

As you know, in the security industry, there has always been discussion over which is better, stateful packet filters or full application proxy firewalls and in recent years both firewalls have moved towards dpi. Packet filters were seen as faster but could be easily bypassed (e.g. a nmap ACK scan used to passed packet filter firewalls because the packet filter assumed a packet with an ACK was part of an already-established session), whereas proxy/application firewalls were slower but more application (layer 7) aware.

Traditionally (network) firewalls would have simply looked at the headers (e.g. IP, TCP, UDP) but not the actual data of the packets, whereas IDS/IPS solutions such as Snort, commercial products (and now Suricata) can perform full packet/protocol analysis.

Additionally, to a degree, it is possible to slot a (DPI) firewall onto the network, give it some basic configuration and leave it running, however, in my limited experience a WAF always needs a lot of love and care, otherwise all hell can break loose (one particular company turned it off for maintenance and forgot to turn it back on, promptly getting hacked).

Taking a look at the Snort configuration file (normally /etc/snort.conf or /etc/snort/snort.conf), one can see configuration for specific preprocessors on TCP streams (e.g. stream4, stream5), fragmentation (e.g. frag2, frag3), HTTP decoding and other protocol anomaly analysis. In my mind, this is true “DPI”. The preprocessors can be thought of as going above and beyond what ‘signatures’ do as they are decoding protocols, packets and looking for anomalies, sometimes with knowledge of what type of server/device is behind the IDS device. Although there have been numerous changes to the preprocessors on Snort over the years, here’s an old but concise article with more detailed explanations on Snort preprocessors and how deep packet inspection works in Snort.

A sample snort rule for examining the HTTP payload for a SQL Injection (SQLi) attack would look like –

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"GPL EXPLOIT xp_cmdshell attempt"; flow:to_server,established; content:"xp_cmdshell"; nocase; classtype:web-application-attack; sid:1061; rev:6;)

There is also a bountiful of documentation on Snort here if you want to read more about it.

Many firewall vendors first began moving to DPI when they added a basic ‘IPS’ to the firewall. As you’d expect, with the initial move to dpi, vendors started off with awareness only a few protocols but this number has gradually increased with time and the application intelligence has improved considerably. As the firewall vendors began to integrate IPS/IDS packet inspection technology, there was much discussion about IDS/IPS/firewalls collapsing into one box but contrary to what some analysts said in 2005, IDS isn't dead yet and many companies run firewalls with separate IDS/IPS devices. It's probably inevitable though that all those technologies will collapse into the one, catch-all security device.

IDS solutions with their ‘dpi’ technology are now turning into Network Security Monitoring (NSM) devices as many tools have been developed to help security engineers understand what’s going on in their network. The excellent, easily-installed, open-source Security Onion Linux-based distro is a great example of a NSM solution and if you want to learn more (about dpi, IDS technology and NSM) please see here.

I was going to do some funky stuff with Security Onion to show how it could be used (at a basic level) to block a sample web application attack, however, Ash beat me to it and posted detecting some basic web application command injection attacks with here.

Just to quickly mention, it is also possible to use Host-Based IDS (HIDS) as part of the overall strategy of protection (in this case against web application attacks). Not surprisingly, Security Onion comes with this also :) in the form of OSSEC, which performs log analysis, file integrity checking and rootkit detection. With regard to a web application attack, it can monitor log files for evidence of XSS, SQLi etc. For example, using a DVWA installation, we see (through the Sguil GUI) OSSEC successfully detecting a XSS attack – 

or in the OSSEC log file (/var/ossec/logs/alerts/alerts.log) -

172.16.15.1 - - [30/Apr/2012:05:51:59 +0000] "GET /dvwa/vulnerabilities/xss_r/?name=%3Cscript%3Ealert%28%22hello%22%29%3B%3C%2Fscript%3E HTTP/1.1" 200 1698 "http://172.16.15.186/dvwa/vulnerabilities/xss_r/?name=%3Cscript%3Ealert%28%22hello%22%29%3C%2Fscript%3E" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.165 Safari/535.19"

OSSEC (which probably requires a blog post of its own) is configured to monitor entries in /var/log/apache2/access.log and so detects the XSS entry.

There are also some SQLi checks performed on the Apache access log. I added a few of my own to the /var/ossec/rules/web_rules.xml file so that OSSEC would detect a basic SQLi (1=1 type) attack –

and the OSSEC log shows –

172.16.15.1 - - [30/Apr/2012:07:34:01 +0000] "GET /dvwa/vulnerabilities/sqli/?id=2+or+x%3Dx%3B--&Submit=Submit HTTP/1.1" 200 1706 "http://172.16.15.186/dvwa/vulnerabilities/sqli/?id=2+or+a%3Da%3B--&Submit=Submit" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.165 Safari/535.19"

Anyway I digress, back to the original question J With regard to a direct comparison (as requested by the person asking the question) –

• A dpi-type device (firewall/IDS/IPS) will know what valid HTTP is, what FTP should look like etc. but it will not know what is appropriate (data) for your web app, i.e. it will not understand the application logic.

• A dpi-type device will most likely be able to detect some basic cross-site scripting (XSS) and SQL Injection attacks (such as looking for '1=1' in the URL or POST body), but it will not be able to do this to the level of a WAF (in understanding command injection, XSS and XSRF attacks), i.e. it will not 'KNOW' the application . However, it can detect something like Skype (or other protocol anomalies) being tunnelled through HTTP, for example.

• As the WAF learns the application and its logic, the WAF can then use this knowledge to prevent unknown attacks as it detects strange traffic (e.g. in the response from the web-server) that is anomalous to the normal traffic pattern.

• All (leading) WAFs typically come with a negative security model also, as they employ signature-based detection functionality (for example, in my old company, our WAF used the Snort signatures from Sourcefire).

• Additionally, many WAFs will perform basic detection on non-standard, non-compliant traffic by default, thus saving a subsequent CPU or memory hit. Below is a list of such non-compliant traffic that will be quite often be reset –

o   HTTP 0.9 version

o   missing Host header

o   malformed HTTP request

o   RFC strict checks for HTTP compliance

o   Content-Length required for POST method

• Many application-layer attacks can bypass dpi-type devices at layer 7 easier than a WAF, though this is not surprising as I’ve said previously, they have different roles and in truth, complement each other in a layered, defence-in-depth secure approach to infrastructure design.

• Typically, a firewall/IDS/IPS device will see all network traffic as they’re deployed inline, whilst a WAF is installed essentially as a proxy in front of the web services layer. If the traffic is encrypted, it will be decrypted (and possibly re-encrypted) on the WAF, whereas all SSL traffic typically travels encrypted through the firewall, IDS and IPS devices.

It is important to note that a web app firewall is not a silver bullet for protecting your web application and there are many researchers constantly looking for new ‘WAF bypass’ attacks. Therefore, the WAF should complement a secure development life cycle such as the secure development principles outlined by David Rook here and the Owasp folk here. Another useful list of recommendations (though it’s taken from a slightly different viewpoint) is the top 25 programming errors by Sans, which can be found here.

On a side note, if you’re interested, I’ve previously presented at an Owasp Ireland Chapter meeting on WAF Implementation.
--
P.S. Other than the OSSEC element and screenshots, this was my actual email answer.