ABI mismatch when building go project with a yocto recipe

Question

I'm trying to write a Yocto recipe to build a Go lang project for the x86_64 target platform (rocko branch). My yocto recipe builds the Go project fine, resolving dependencies with dep tool. But the binary built doesn't run on the target platform. When trying to execute it on the target, I get this error:

$ /usr/bin/mybin
abi mismatch detected between the executable and libstd.so
fatal error: abi mismatch
runtime: panic before malloc heap initialized

This is my custom recipe to build the project:

GO_IMPORT = "bitbucket.org/xxx/myproject"
SRC_URI = "git://${GO_IMPORT}/protocol=http;user=${GIT_USER}:${GIT_PASS};destsuffix=${PN}-${PV}/src/${GO_IMPORT}"
SRCREV = "7777ee7777777c9777774bb777780777759d777771777"

CGO_ENABLED = "0"

inherit go

do_compile_prepend() {
    rm -f ${WORKDIR}/build/src/${GO_IMPORT}/Gopkg.toml
    rm -f ${WORKDIR}/build/src/${GO_IMPORT}/Gopkg.lock
    cd ${WORKDIR}/build/src/${GO_IMPORT}
    dep init
    dep ensure
}

do_install_append() {
    rm -f ${D}/usr/bin/dep
}

deltask do_compile_ptest_base
deltask do_compile_ptest

DEPENDS = "go-dep-native mercurial-native"

INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
INHIBIT_PACKAGE_STRIP = "1"

RDEPENDS_${PN}-staticdev += "bash"
RDEPENDS_${PN}-dev += "bash"

What should be the proper way to manage this issue?

score 3 · Accepted Answer · answered Mar 08 '18 at 19:40

Ok, so I found the a better way of prevent this problem and also more information about the core problem.

Better Solution

In your recipe add the following line:

GO_LINKSHARED = ""

This will prevent -builmdmode=shared and do a normal go build instead of linking against a standard library. Funny thing, by removing the linking to the standard library I only added 100KB to my bianry size and libstd.so is 30MB.

Core Problem

The core problem is that when you are using the default shared buildmode, when you build a go binary you also build the go runtime libstd.do in another rpm package. In theory, if your target has the same runtime library version then you do not need to install the runtime again. This however does not appear to be the case. Go hashes all the package definitions it puts into libstd and then hashes those hashes in order to create the ABI hash. Two instances of libstd.so of the same go version should have the same hash, because the have the same functions, but that does not appear to be the case. I wrote a more technical bug report on yocto's bug tracker here.

Very nice bugfix. Thanks a lot! I read the bug report and it contais really valuable information. Hopefully it will be fixed soon. Thanks a lot for your time investigating this issue, Mr Liam Kelly! :-) — aicastell, Mar 09 '18 at 08:45
I was getting really strange SIGSEGV issues, using the sumo branch of poky with go 1.10.3 -- cross-compiling for armv6 (raspberrypi0-wifi target) Setting `GO_LINKSHARED = "" ` totally fixed the issue. Thanks! — bvarner, Oct 07 '18 at 20:46

aicastell · Answer 2 · 2018-02-06T09:27:11.660

I can provide more information regarding this issue:

$ ldd /usr/bin/mybin
    linux-vdso.so.1 (0x00007ffe321d9000)
    libstd.so => /usr/lib/go/pkg/linux_amd64_dynlink/libstd.so (0x00007f95463d7000)
    libc.so.6 => /lib/libc.so.6 (0x0000003436200000)
    libdl.so.2 => /lib/libdl.so.2 (0x0000003436a00000)
    libpthread.so.0 => /lib/libpthread.so.0 (0x0000003436600000)
    /lib/ld-linux-x86-64.so.2 (0x0000003435e00000)

$ file /usr/bin/mybin
/usr/bin/mybin: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=0103acf63634df77e600eb114bf59977462ca51d, with debug_info, not stripped

$ file /usr/lib/go/pkg/linux_amd64_dynlink/libstd.so
/usr/lib/go/pkg/linux_amd64_dynlink/libstd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=2d2f135b2f9eab19a7ceff1923820bf635e10eb1, with debug_info, not stripped

Problem seems related with the generation of dynamic library libstd.so (controlled by GO_DYNLINK). The generation of that library is managed by this fragment of source code defined on poky/meta/recipes-devtools/go/go-runtime.inc:

do_compile() {
    export GOBIN="${B}/bin"
    export CC="${@d.getVar('BUILD_CC').strip()}"
    rm -rf ${GOBIN} ${B}/pkg
    mkdir ${GOBIN}
    cd src
    ./make.bash --host-only
    cp ${B}/pkg/tool/${BUILD_GOTUPLE}/go_bootstrap ${B}
    rm -rf ${B}/pkg/${TARGET_GOTUPLE}
    ./make.bash --target-only
    if [ -n "${GO_DYNLINK}" ]; then
        cp ${B}/go_bootstrap ${B}/pkg/tool/${BUILD_GOTUPLE}
        GO_FLAGS="-buildmode=shared" GO_LDFLAGS="-extldflags \"${LDFLAGS}\"" ./make.bash --target-only
    fi
    cd ${B}
}

I did a test, and after commenting this condition:

    # if [ -n "${GO_DYNLINK}" ]; then
    #    cp ${B}/go_bootstrap ${B}/pkg/tool/${BUILD_GOTUPLE}
    #    GO_FLAGS="-buildmode=shared" GO_LDFLAGS="-extldflags \"${LDFLAGS}\"" ./make.bash --target-only
    # fi

My recipe generates a working mybin linked as static.

So, after providing all this information, I have more questions: what is the proper way to set this variable GO_DYNLINK from my custom recipe? Because I have tried setting it in different ways:

GO_DYNLINK=""
unset GO_DYNLINK
GO_DYNLINK="0"

And none of them works (again the ABI mismatch problem). Thanks!! :)

The values are hard coded in `goarch.bbclass`. I found the only way to get around this is to make a `go-runtime_%.bbappend` file and copy your version of the `do_compile` function. — Liam Kelly, Mar 07 '18 at 20:48
It works as suggested. Thank you very much Mr. Liam Kelly! :-) — aicastell, Mar 08 '18 at 12:11

ABI mismatch when building go project with a yocto recipe

2 Answers2