How to cross-compile a Go package using github.com/mattn/go-sqlite3 for a Google Cloud VM?

Question

This question is the same as go-sqlite3 compiler arguments for cross compile OSX to linux, but since that one doesn't have an answer I'll try asking again. I have a Go package which uses the github.com/mattn/go-sqlite3 SQLite3 driver, similar to the following:

package main

import (
    _ "github.com/mattn/go-sqlite3"
)

However, if I try to compile this for Linux AMD64 with CGO enabled (as required by SQLite3), I get the following errors:

> env GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build
# github.com/mattn/go-sqlite3
sqlite3-binding.c:33123:42: error: use of undeclared identifier 'pread64'
sqlite3-binding.c:33141:42: error: use of undeclared identifier 'pwrite64'
sqlite3-binding.c:33274:22: error: invalid application of 'sizeof' to an incomplete type 'struct unix_syscall []'
sqlite3-binding.c:33283:22: error: invalid application of 'sizeof' to an incomplete type 'struct unix_syscall []'
sqlite3-binding.c:33310:20: error: invalid application of 'sizeof' to an incomplete type 'struct unix_syscall []'
sqlite3-binding.c:33327:16: error: invalid application of 'sizeof' to an incomplete type 'struct unix_syscall []'
sqlite3-binding.c:14220:38: note: expanded from macro 'ArraySize'
sqlite3-binding.c:33331:14: error: invalid application of 'sizeof' to an incomplete type 'struct unix_syscall []'
sqlite3-binding.c:14220:38: note: expanded from macro 'ArraySize'
sqlite3-binding.c:35984:11: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
sqlite3-binding.c:33127:49: note: expanded from macro 'osPread64'
sqlite3-binding.c:36096:17: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
sqlite3-binding.c:33145:57: note: expanded from macro 'osPwrite64'

Similarly, if I specify --tags "linux" as described at https://github.com/mattn/go-sqlite3#linux, I get these errors:

> go build --tags "linux"
# runtime/internal/sys
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:8:7: GOOS redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:7:14
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:10:7: GoosAix redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:9:17
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:11:7: GoosAndroid redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:10:21
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:12:7: GoosDarwin redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:11:20
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:13:7: GoosDragonfly redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:12:23
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:14:7: GoosFreebsd redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:13:21
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:15:7: GoosHurd redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:14:18
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:16:7: GoosJs redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:15:16
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:17:7: GoosLinux redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:16:19
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:18:7: GoosNacl redeclared in this block
    previous declaration at /usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_darwin.go:17:18
/usr/local/Cellar/go@1.12/1.12.12/libexec/src/runtime/internal/sys/zgoos_linux.go:18:7: too many errors
# runtime/cgo
duplicate symbol '__cgo_sys_thread_start' in:
    $WORK/b040/_x004.o
    $WORK/b040/_x007.o
duplicate symbol '_x_cgo_init' in:
    $WORK/b040/_x004.o
    $WORK/b040/_x007.o
ld: 2 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

How can I cross-compile this package on my MacOS (Darwin AMD64) laptop to run on a Google Cloud Linux VM?

cgo will be linkiing against the libraries it uses on your Mac. So it won't work. You'd have to replicate all the cgo dependencies that would exist on linux and put them on your mac. Have fun with that. Instead, create a Linux VM and compile on that. — Sean F, Jan 12 '20 at 01:02

score 7 · Answer 1 · answered Jan 12 '20 at 01:09

7

It turns out from https://github.com/mattn/go-sqlite3/issues/384#issuecomment-433584967 that this can be solved by using the musl-cross cross-compiler toolchain:

brew install FiloSottile/musl-cross/musl-cross

followed by

> env CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 \
  go build -ldflags "-linkmode external -extldflags -static"

answered Jan 12 '20 at 01:09

Kurt Peek

52,165
91
301
526

2

As an update: if you want to do this from an M1 Mac to linux, then use this toolchain: https://github.com/messense/homebrew-macos-cross-toolchains Use `x86_64-unknown-linux-gnu-gcc` and `x86_64-unknown-linux-gnu-g++` as your compilers. – poundifdef May 27 '21 at 22:18

How to cross-compile a Go package using github.com/mattn/go-sqlite3 for a Google Cloud VM?

1 Answers1