I'm attempting to add the Openssl FIPS module to an existing embedded powerpc system running Linux. Its mostly going well, but I'm stuck on an issue that occurs during the linking phase. If I don't disable certain ciphers when configuring openssl, then I get linking errors like so:
passwd.o: In function `do_passwd':
passwd.c:(.text+0x70c): undefined reference to `DES_crypt'
genrsa.o: In function `genrsa_main':
genrsa.c:(.text+0x528): undefined reference to `EVP_des_cbc'
gendsa.o: In function `gendsa_main':
gendsa.c:(.text+0x3d8): undefined reference to `EVP_des_cbc'
speed.o: In function `speed_main':
speed.c:(.text+0x1078): undefined reference to `private_DES_set_key_unchecked'
speed.c:(.text+0x108c): undefined reference to `private_DES_set_key_unchecked'
speed.c:(.text+0x10a0): undefined reference to `private_DES_set_key_unchecked'
speed.c:(.text+0x1d1c): undefined reference to `DES_ncbc_encrypt'
speed.c:(.text+0x1e80): undefined reference to `DES_ede3_cbc_encrypt'
speed.c:(.text+0x3ec4): undefined reference to `EVP_mdc2'
speed.c:(.text+0x401c): undefined reference to `DES_options'
In this case, I get these errors unless I specify "no-des" in the configuration options, but the same issue applies to other ciphers as well, such as cast, seed and rc2 (among others).
Some of the existing codebase references some of these cipher functions (especially DES), so removing them entirely isn't ideal. My understanding is that if FIPS mode were enabled, then use of these non-FIPS ciphers would be prevented by the library at run time (by generating errors, I presume), not that all the non-FIPS ciphers need to be disabled at compile-time.
Here's how I'm configuring both openssl-fips and openssl:
openssl-fips:
MACHINE=ppc \
RELEASE=2.6.11 \
SYSTEM=linux2 \
ARCH=ppc \
CROSS_COMPILE="powerpc-405-linux-gnu-" \
HOSTCC=gcc \
./config
openssl:
CC="gcc [snipped]" \
FIPSDIR=$(ROOTPREFIX)$(PREFIX)/usr/local/ssl/fips-2.0 \
./Configure \
linux-ppc \
no-idea \
no-rc5 \
no-mcs2 \
no-hw \
no-krb5 \
no-seed \
no-rc2 \
no-bf \
no-cast \
--prefix=/usr \
threads \
shared \
fips \
--cross-compile-prefix=powerpc-405-linux-gnu-; \