Bit9: FIPS Compliance May Weaken OpenSSL

A security researcher has poked holes in an upcoming release of a FIPS-based OpenSSL version by arguing that it is insecure before it is even available.

In a blog, Dan Brown of Bit9, an adaptive whitelist security vendor, explains that the Federal Information Processing Standard (FIPS) library consists of two parts, a FIPS module (canister) and the API. However, these two parts are built from two different versions of the source code. This is, in part, so the code can be checked against the other to see whether there's been tampering.

Brown writes:

After configuration, a stub executable must be built that includes the FIPS canister, the purpose of which is to hash the in-memory image of the FIPS canister and output the hash to standard output. This is put into a C source file as data to be included in the final DLL or shared object product eventually produced. At run time, FIPS mode is entered via a special call, which calculates the in-memory hash of the FIPS canister and compares it to the original hash generated by the stub executable at build time. If these match, all is good and FIPS mode is enabled

But his point, that a FIPS-based OpenSSL is insecure, involves the fact that the code is not re-locationable within memory. Meaning, if OpenSSL uses Address Space Layout Randomization (ASLR), then it will fail the FIPS check since the underlying hash value will be different.

Too technical? Brown offers this. Last week's vulnerability in OpenSSL was fixed, however, the version undergoing FIPS certification now is probably the unpatched, vulnerable version. So you can have a FIPS compliant OpenSSL, but it might not be secure.