How to link MacOS x86_64 libraries to Universal apps?

Question

My MacOS app uses multiple statically linked SDKs (C / C++ libraries) from multiple hardware manufacturers. Since the libraries are mostly compiled for x86_64 and the manufacturers are slow in adapting to arm64, I have to also build my project for the x86_64 architecture exclusively (as soon as there is only one x86_64 library that you need to include, the whole project won't build for arm64 / Universal). With Rosetta 2 this works, but of course does not give full potential performance on Apple Silicon, specifically when you are looking for its additional features (GPUs).

Is there any way to link a x86_64 library while having the rest of the application in Universal? According to Apple docs the answer is no (https://developer.apple.com/documentation/apple-silicon/porting-your-macos-apps-to-apple-silicon), but... Is it possible to use Rosetta 2 to pre-translate the library into arm64 / Universal binary and then link it to the rest of the app? Any other creative idea on how to solve this? Otherwise it seems I will have to write my own version of that SDK.

Answer 1

Let's summarize what I have learnt so far:

In order to build a universal binary, all libraries need to be available in universal. Let's say you have 20 SDKs that you want to include and only 10 of them are universal binaries, what do you do?

The reason there are so many libraries in my app comes from the fact that the app supports many manufacturers and the user may choose from these.

Quick fix

Therefore, I have decided to mock the unavailable arm64 libraries and making that visible to the user: If users want to use the arm64 version of the app, they just won't be able to access the devices with non-universal SDKs. If they need support for these devices, they will need to start the app with Rosetta (click on the app executable, show info, select Open using Rosetta), thereby making it transparent which manufacturers are supporting the arm64 platform and which are not.

How do you mock a arm64 version of a x86_64 library? Create a C file and implement empty functions that return an error code, for all functions that you use. Build for arm64 and combine with the actual x86_64 library using the lipo tool.

Example:

#include "AbcSDK.h"

int AbcGetNumOfConnectedDevices() {
    return 0;
}

ABC_ERROR_CODE AbcGetDeviceInfo(ABC_DEVICE_INFO *pAbcDeviceInfo, int iDeviceIndex) {
    return ABC_ERROR_INVALID_INDEX;
}
...

Makefile:

CC=cc
AR=ar
LIPO=lipo

DEPS = include/AbcDeviceSDK.h
LIB_x64 = lib/x64/libAbcDeviceSDK.a
OBJ = AbcDeviceSDK.o

libAbcDeviceSDK.a: AbcDeviceSDK-arm64.a $(LIB_x64)
    $(LIPO) $^ -create -output libAbcDeviceSDK.a
    cp libAbcDeviceSDK.a YOURPROJECT/Libraries/AbcDeviceSDK.a

%.o: %.c $(DEPS)
    $(CC) -c -o $@ $< 

AbcDeviceSDK-arm64.a: $(OBJ)
    $(AR) rcs $@ $^ 

clean:
    rm -f *.o *.a

Of course, this does not solve the original problem, but it does at least build universal binaries.

Rosetta ahead-of-time translation

While Rosetta does save pre-translated binaries into /var/db/oah as .aot files and these files actually contain arm64 code, they are not stand alone (they do call the Rosetta runtime). See https://ffri.github.io/ProjectChampollion/ for further reference. This seems to be a dead-end.

Decompile using Ghidra

Ghidra is a great open-source reverse-engineering tool that you can use to understand the library (or any piece of software). It gives you two views into the code that you look at in parallel:

1. assembly
2. pseudo C

Ghidra is great to iteratively bring the automatically decompiled C code from a state of unnamed functions, variables, wrong types into something more akin to how it would have looked like before compilation. The resulting code will not compile with a compiler, it is meant as a more readable view into the assembly code. Be prepared to invest weeks and bring curiosity of how things work under the hood.

So to accomplish my task, I needed to first understand the code fully with Ghidra and then rewrite the library from scratch in C (or whatever language I prefer). With small libs this might be doable.