2

I am in the process of migrating an IPsec application from VxWorks to Linux. And I am pretty new to Linux. The component that uses IPsec is a 32-bit application. However, the entire product runs in a 64-bit environment.

We are using Ubuntu 16.04 (LTS) on a X86_64 machine. Linux kernel version is 4.9.

32-bit IPsec on 64-bit kernels has compatibility issues with respect to structure padding in xfrm.h. This was an easy problem to fix. However, after fixing this issue, I was not able to install SAs. The XFRM framework was returning -95 not supported errors.

After debugging I found the check below in xfrm_user_rcv_msg() was causing the issue. I commented the block just for testing. After rebuilding the kernel with this change the 32-bit IPsec app worked fine on the 64-bit kernel.

I wanted to understand the reason behind this check and if it really is necessary to remove it to run 32-bit IPsec applications on 64-bit kernels.

#ifdef CONFIG_COMPAT 
    if (in_compat_syscall())
        return -EOPNOTSUPP;
#endif

Regards Jayalakshmi

ecdsa
  • 542
  • 3
  • 12
user50392
  • 21
  • 2

1 Answers1

1

This check was added with the 4.1 kernel to prevent issues with incompatible XFRM requests from 32-bit userland programs (the assumption is obviously that they were built against an unmodified xfrm.h header). Here is the original commit information:

commit 74005991b78a0a7a6546004fb37d33a651c613e7
Author: Fan Du <fan.du@intel.com>  Tue Jan 27 10:00:29 2015
Committer:  Steffen Klassert <steffen.klassert@secunet.com>  Tue Mar  3 10:10:16 2015

xfrm: Do not parse 32bits compiled xfrm netlink msg on 64bits host

structure like xfrm_usersa_info or xfrm_userpolicy_info
has different sizeof when compiled as 32bits and 64bits
due to not appending pack attribute in their definition.
This will result in broken SA and SP information when user
trying to configure them through netlink interface.

Inform user land about this situation instead of keeping
silent, the upper test scripts would behave accordingly.

Signed-off-by: Fan Du <fan.du@intel.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
ecdsa
  • 542
  • 3
  • 12
  • Does this means no 32 bit IPsec application run on 64 bit kernel? I read on strongswan mailing list about 32 bit strongswan library having the similar issue on 64 bit kernel. After doing modification to xfrm.h 32 bit strongswan worked successfully on 64 bit kernel. I believe COMPAT layer takes care of converting 32 bit application data to be in compatible with 64 bit kernel. I could see XFRM policies after I modified xfrm.h. – user50392 Feb 08 '18 at 14:11
  • Yes, since the 4.1 kernel that's the case. The reasoning probably was that you usually don't install 32-bit software that is compiled against a modified xfrm.h header but a 32-bit binary package from your distribution that was compiled against the original xfrm.h. This check avoids strange result if such software is used, but as you saw it also prevents you from using software that's explicitly compiled against a modified version of xfrm.h. – ecdsa Feb 08 '18 at 14:56
  • By the way, only the XFRM interface is affected. 32-bit IPsec software that uses the PF_KEYv2 interface (e.g. strongSwan with the _kernel-pfkey_ plugin) probably works fine on 64-bit kernels. – ecdsa Feb 08 '18 at 15:07