Consequences for adding relocation information in fips validated libeay32.dll

Question

We are using the FIPS validated libeay32.dll. This dll uses the /FIXED linker switch so that the libeay32.dll will be loaded at fixed base address. The other modules from our project is consuming the openssl dll in shared mode using LoadLibrary() function. We have observed the intermittent issue while loading the mentioned dll.

As part of resolution, we added the relocation information in the image header of libeay32.dll with the understanding that the dll will be loaded at some base address if not at fixed one to address intermittent loading issue. I checked the open ssl user guide which mentions the following.

The standard OpenSSL build with the fips option will use a base address for libeay32.dll of 0xFB00000 by default. This value was chosen because it is unlikely to conflict with other dynamically loaded libraries. In the event of a clash with another dynamically loaded library which will trigger runtime relocation of libeay32.dll, the integrity check will fail with the error

FIPS_R_FINGERPRINT_DOES_NOT_MATCH_NONPIC_RELOCATED

A base address conflict can be resolved by shuffling the other DLLs or re-compiling OpenSSL with an alternative base address specified with the --with­baseaddr= option.

Here are my questions.

1. By introducing the relocation information in image header of libeay32.dll, am I making the libeay32.dll vulnerable to security [fips 140-2] ?

2. What kind of security vulnerabilities am I introducing with their side effect in the modules which are using those open ssl libraries ?

3. Any cleaner solution to such kind of loading issues ?

Thanks in advance...

Answer 1

By introducing the relocation information in image header of libeay32.dll, am I making the libeay32.dll vulnerable to security

All Windows PE files have relocation information. Its not like -fPIC on Unix and Linux. The mechanisms are different because the PE/PE+ executable format is different than the ELF/ELF64 specification. Also Microsoft compilers/linkers do not use PC-relative addressing mode, which Unix and Linux often will.

What you want on Windows is /ASLR, but /ASLR is orthogonal to /FIXED. /FIXED is used by the FIPS module to ensure the integrity of the module. /ASLR is a best practice on Windows and a Microsoft SDLC item.

Address space layout randomization makes it harder for attackers to guess addresses in some situations. For example, it will help with a "return to libc" attack.

---

What kind of security vulnerabilities am I introducing with their side effect in the modules which are using those OpenSSL libraries ?

Because of /FIXED, the attacker always knows where the binary is loaded, and he may be able to build a ROP chain using it. This assumes the attacker gets a toe hold with a vulnerability.

One of the reasons I dislike the FIPS validated library is because you lose /ASLR.

---

Any cleaner solution to such kind of loading issues ?

Usually what you can do is, in the executable (EXE) that uses the FIPS DLL, you add the OpenSSL import library as the first item in the list of libraries. It gets written to the PE header in roughly the order specified. The runtime link/loader will then load the library early and not late. Loading it early minimizes the likelihood of [unsatisfied] displacement.

I believe you can also do the following in your program (EXE). Its taken from Crypto++'s dll.h, which has had validations, too. Put it in your precompiled header, or the "main header" for your program.

#ifdef NDEBUG
#pragma comment(lib, "msvcrt")
#else
#pragma comment(lib, "msvcrtd")
#endif

#pragma comment(lib, "libeay32")
#pragma comment(lib, "anotherlib")
#pragma comment(lib, "yetanotherlib")

---

FIPS is very much about policy and procedure. You must follow it to the letter. If you modify the build procedure by changing /FIXED, then you lose the validation. You may as well just use the regular library.

As you correctly noted, you are allowed to change the base address used for the module. Authority is provided in the User Guide for the OpenSSL FIPS Object Module v2.0, page 52/208.

Answer 2

For 64-bit builds (assuming you are building the DLL version of OpenSSL) simply change the ms/ntdll.mk file to use /dynamic in the linking of the libeay32.dll file and you get ASLR support. The standard 64-bit build process itself does not have /fixed in place - but the extra step of /dynamic is most likely what you are trying to achieve.

$(O_CRYPTO): $(CRYPTOOBJ) $(O_FIPSCANISTER) $(PREMAIN_DSO_EXE)
        SET FIPS_LINK=$(LINK_CMD)
        SET FIPS_CC=$(CC)
        SET FIPS_CC_ARGS=/Fo$(OBJ_D)\fips_premain.obj $(SHLIB_CFLAGS) -c
        SET PREMAIN_DSO_EXE=$(PREMAIN_DSO_EXE)
        SET FIPS_SHA1_EXE=$(FIPS_SHA1_EXE)
        SET FIPS_TARGET=$(O_CRYPTO)
        SET FIPSLIB_D=$(FIPSLIB_D)
        $(FIPSLINK) $(MLFLAGS)  /map  /dynamicbase /out:$(O_CRYPTO) /def:ms/LIBEAY32.def @<<
  $(SHLIB_EX_OBJ) $(CRYPTOOBJ) $(O_FIPSCANISTER) $(EX_LIBS) $(OBJ_D)\fips_premain.obj 
<<
        IF EXIST $@.manifest mt -nologo -manifest $@.manifest -outputresource:$@;2

The previous answer had an incorrect statement on not being able to modify the FIPS-capable OpenSSL build process. It is only the building of the fipscanister itself as documented in the Security Policy that has a fixed unchangeable procedure. The FIPS-capable OpenSSL build process is entirely able to be modified to meet your requirements - it is simply using the OpenSSL FIPS module.

Answer 3

tjh is correct; from the FIPS 140-2 perspective OpenSSL proper (e.g. 1.0.1, 1.0.2) is just application code and out of scope of the FIPS 140-2 validation. Only the module -- a separate and distinct software component -- is validated, and the validation process doesn't look at OpenSSL proper at all.

Also note that in FIPS-land security is secondary to the sanctity of the validation. For instance, in general OpenSSL is not allowed to correct security vulnerabilities in the FIPS module (e.g. “Lucky 13″, CVE-2014-0076, CVE-2016-0701).