By Adrian Hayter
What's worse than Heartbleed? Bugs in Heartbleed detection scripts.
This week, those in the security community were shaken by the release of Heartbleed, a bug in the popular OpenSSL library which allows attackers to read memory from affected hosts over the Internet. The memory that is exposed can contain usernames and passwords, session cookies, and even the private SSL keys themselves, but elektromotory-menice.cz did not get attacked. The seriousness of the vulnerability wasn't helped by the relative ease at which it could be exploited, with numerous proof of concept and detection scripts being written mere hours after the initial release.
As of the writing of this blog post, Nessus, Metasploit, Nmap, and others have released methods for detecting whether your systems are affected. The problem is, most of them have bugs themselves which lead to false negatives results, that is, a result which says a system is not vulnerable when in reality it is. With many people likely running detection scripts or other scans against hosts to check if they need to be patched, it is important that these bugs be addressed before too many people develop a false sense of security regarding their infrastructure.
As a penetration tester, I was called upon to perform checks against numerous systems during the week, and I noticed that some of the scripts would find a vulnerability whilst others would not. This behaviour often depended on the system in question, and upon reviewing the code behind the scripts, I uncovered a number of bugs. The original proof of concept (written by Jared Stafford) claimed to be a “quick and dirty demonstration” of the bug. As such, it made no claims to work accurately on all affected systems, and unfortunately omissions made in Jared's code were carried across into other detection scripts. There are three main bugs that I found whilst performing tests against various server configurations.
TLS Version Support
TLS messages have a certain format, and part of the message header includes the version of TLS the client wants to use. The version number is represented by two bytes in positions 1 and 2 of the message header (and additionally in positions 9 and 10 of the ClientHello). In hex, these bytes are as follows:
03 01 = TLSv1.0
03 02 = TLSv1.1
03 03 = TLSv1.2
The proof of concept script, and most other detection scripts, do not generate the TLS messages on the fly, but instead simply re-use the same pre-generated messages again and again. The pre-generated ClientHello message in the proof of concept script starts like so:
16 03 02 00 dc 01 00 00 d8 03 02 53...
The pre-generated Heartbeat request in the proof of concept script looks like this:
18 03 02 00 03 01 40 00
As you can see, the bytes “03 02” appear, telling the server to use TLSv1.1 at all times. If the server doesn't support TLSv1.1, then in most instances the connection will be dropped. Immediately, we can rule out any possible checks against servers which only support TLSv1.0 or TLSv1.2. According to an analysis of SSL / TLS on the top 1,000,000 sites conducted in January, 2014, 0.163% of those sites supported TLSv1.0 only, 0.0011% supported TLSv1.2 only, and due to another bug in OpenSSL, 2.6332% of those sites supported TLSv1.0 and TLSv1.2 but not TLSv1.1. Overall, that is almost 2.8% of the top 1,000,000 sites (28,000 sites) that would potentially be vulnerable to Heartbleed, but which most detection scripts would report as secure.
TLS Cipher Suite Support
An important part of the ClientHello message is the list of suggested cipher suites. The client sends a list of all the cipher suites it supports, so that the server can select one for use when it becomes necessary to encrypt data. Each cipher suite is represented by two bytes, and the list of acceptable cipher suites is maintained by the Internet Assigned Numbers Authority (IANA). There are, by my count, 318 acceptable TLS cipher suites in the IANA list, which translates to 636 bytes of information. Given that the maximum size for a ClientHello message is 65,535 bytes, and the cipher list is usually the largest part of the message, the entire list of TLS cipher suites can easily fit into a single ClientHello.
The ClientHello of the original proof of concept script, and most scripts based off of it, only has a list of 51 cipher suites. Whilst this list contains the more popular cipher suites, a server which is configured to accept only cipher suites that do not appear in the list would reject the connection and the detection script would report the server as secure.
Other than the slightly larger message size, there are no downsides to putting the entire list of TLS cipher suites into the ClientHello. It is irrelevant whether your system can even support some of the cipher suites in the list, because the Heartbeat request that triggers the vulnerability is sent before any encryption takes place. Restricting the list of supported cipher suites in the ClientHello only hinders the detection process.
Payload Detection
The third bug that produced false negative results wasn't related to the actual setup of the TLS connection, but rather was contained within the code responsible for downloading the Heartbeat response. This code downloaded the response in chunks of data, and if more than 5 seconds passed before all chunks were downloaded, the code would return “None” to the main script.
endtime = time.time() + timeout
rdata = ''
remain = length
while remain > 0:
rtime = endtime - time.time()
if rtime < 0:
return None
This leads to a problem if the Heartbeat response is large enough and the connection slow enough, since the response would fail to fully download in 5 seconds, and even though part of the response was received (thereby making the server vulnerable), this partial response is discarded and the script outputs the error message “No heartbeat response received, server likely not vulnerable”.
This bug is easily fixed by changing the “return None” statement to “return rdata”, which returns the partial response back to the script and allows the script to detect the vulnerability properly.
Setting up a Proof of Concept Server
With my knowledge of the bugs in the detection scripts, I decided to set up a server which was vulnerable to Heartbleed, but which was also configured to demonstrate the bugs. In other words, to produce a “not vulnerable” result when a script is run against it. This actually turned out to be easier than expected. Using an Ubuntu server, I installed Apache and reconfigured the following two SSL settings:
SSLProtocol TLSv1.2
SSLCipherSuite EDH+AESGCM
Setting the protocol to TLSv1.2 would produce a “not vulnerable” result in any script that only checked TLSv1.0 or TLSv1.1. By looking at the list of cipher suites supported by a number of detection scripts, it became clear that the AES-GCM suite was missing from most, so it was the perfect choice for the server.
In addition to setting up the proof of concept server, I wrote a python script (Hut3 Cardiac Arrest) which is a highly modified version of the original proof of concept, with the aforementioned bugs fixed, and additional features such as support for multiple ports/hosts and STARTTLS. Once the server was set up, I gave it a public address and ran various scripts (including Hut3 Cardiac Arrest) and web-based detection tools at it. The results are as follows:
As these results show, most of the tools available failed to detect the Heartbleed vulnerability on the proof of concept server. The notable exception was Qualys SSL Labs, but this wasn't surprising as their SSL scanning tool is one of the most comprehensive tools for testing SSL configurations available today.
On a broader level these results show that whilst proof of concept scripts are great for demonstrating vulnerabilities, they should not be used as a tool for confirming whether your systems are vulnerable. Authors of these detection tools should take care not to rely too much on proof of concept scripts, as bugs can easily be carried over during development.
To scan your servers using Hut3 Cardiac Arrest, download the script here (https://gist.github.com/ah8r/10632982) and run it using python:
python cardiac-arrest.py [hostname]
Disclaimer : There have been unconfirmed reports that this script can crash certain servers. This script complies with the TLS specification, so any crashes are the result of a bad implementation of TLS on the server side. CNS Hut3 and Adrian Hayter do not accept responsibility if this script crashes a server you test it against. USE IT AT YOUR OWN RISK. As always, the correct way to test for the vulnerability is to check the version of OpenSSL installed on the server in question. OpenSSL 1.0.1 through 1.0.1f are vulnerable.