Using Wasm with data-fusion

Question

I am very beginner using Rust but I am trying to create a project to implement Webassembly and using a library called data-fusion, the problem is the compilation fails with the message:

error: failed to run custom build command for `zstd-sys v1.6.3+zstd.1.5.2`

FULL ERROR:

    The following warnings were emitted during compilation:

warning: In file included from zstd/lib/common/entropy_common.c:18:
warning: In file included from zstd/lib/common/mem.h:24:
warning: zstd/lib/common/zstd_deps.h:29:10: fatal error: 'string.h' file not found
warning: #include <string.h>
warning:          ^~~~~~~~~~
warning: 1 error generated.
warning: In file included from zstd/lib/common/fse_decompress.c:20:
warning: In file included from zstd/lib/common/bitstream.h:29:
warning: In file included from zstd/lib/common/mem.h:24:
warning: zstd/lib/common/zstd_deps.h:29:10: fatal error: 'string.h' file not found
warning: #include <string.h>
warning:          ^~~~~~~~~~
warning: 1 error generated.

error: failed to run custom build command for `zstd-sys v1.6.3+zstd.1.5.2`

Caused by:
  process didn't exit successfully: `/Users/pedro/rust/wasmpack-ounity/helloworld/target/release/build/zstd-sys-da1f85a3ac3fe2ed/build-script-build` (exit status: 1)
  --- stdout
  cargo:rustc-cfg=feature="std"
  TARGET = Some("wasm32-unknown-unknown")
  HOST = Some("x86_64-apple-darwin")
  CC_wasm32-unknown-unknown = None
  CC_wasm32_unknown_unknown = None
  TARGET_CC = None
  CC = Some("/usr/local/opt/llvm/bin/clang")
  CFLAGS_wasm32-unknown-unknown = None
  CFLAGS_wasm32_unknown_unknown = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  running: "/usr/local/opt/llvm/bin/clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=wasm32-unknown-unknown" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/Users/pedro/rust/wasmpack-ounity/helloworld/target/wasm32-unknown-unknown/release/build/zstd-sys-9d373a088b510333/out/zstd/lib/common/entropy_common.o" "-c" "zstd/lib/common/entropy_common.c"
  cargo:warning=In file included from zstd/lib/common/entropy_common.c:18:
  cargo:warning=In file included from zstd/lib/common/mem.h:24:
  cargo:warning=zstd/lib/common/zstd_deps.h:29:10: fatal error: 'string.h' file not found
  cargo:warning=#include <string.h>
  cargo:warning=         ^~~~~~~~~~
  cargo:warning=1 error generated.
  exit status: 1
  running: "/usr/local/opt/llvm/bin/clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=wasm32-unknown-unknown" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/Users/pedro/rust/wasmpack-ounity/helloworld/target/wasm32-unknown-unknown/release/build/zstd-sys-9d373a088b510333/out/zstd/lib/common/fse_decompress.o" "-c" "zstd/lib/common/fse_decompress.c"
  cargo:warning=In file included from zstd/lib/common/fse_decompress.c:20:
  cargo:warning=In file included from zstd/lib/common/bitstream.h:29:
  cargo:warning=In file included from zstd/lib/common/mem.h:24:
  cargo:warning=zstd/lib/common/zstd_deps.h:29:10: fatal error: 'string.h' file not found
  cargo:warning=#include <string.h>
  cargo:warning=         ^~~~~~~~~~
  cargo:warning=1 error generated.
  exit status: 1

  --- stderr


  error occurred: Command "/usr/local/opt/llvm/bin/clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=wasm32-unknown-unknown" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/Users/pedro/rust/wasmpack-ounity/helloworld/target/wasm32-unknown-unknown/release/build/zstd-sys-9d373a088b510333/out/zstd/lib/common/entropy_common.o" "-c" "zstd/lib/common/entropy_common.c" with args "clang" did not execute successfully (status code exit status: 1).


warning: build failed, waiting for other jobs to finish...
The following warnings were emitted during compilation:

warning: liblz4/lib/lz4.c:178:10: fatal error: 'stdlib.h' file not found
warning: #include <stdlib.h>   /* malloc, calloc, free */
warning:          ^~~~~~~~~~
warning: 1 error generated.
warning: liblz4/lib/lz4frame.c:74:10: fatal error: 'stdlib.h' file not found
warning: #include <stdlib.h>   /* malloc, calloc, free */
warning:          ^~~~~~~~~~
warning: 1 error generated.

error: build failed
Error: Compiling your crate to WebAssembly failed
Caused by: failed to execute `cargo build`: exited with exit status: 101
  full command: "cargo" "build" "--lib" "--release" "--target" "wasm32-unknown-unknown"

There is a dependency problem, I tried to include 'zstd-sys = "1.6.3+zstd.1.5.2"' as dependency but error persists.

github repo

Can you share more of the output? The build command it ran should have output a bit more detailed info as to what went wrong. — kmdreko, Jun 11 '22 at 20:46

score 1 · Answer 1 · answered Jun 12 '22 at 03:03

1

Crates that invoke native libraries (i.e. libraries written in C), such as the zstd library, cannot be used in WebAssembly. Usually these crates are postfixed with -sys. I'm not familiar with the data-fusion crate, but I'm guessing that it does not support WebAssembly.

answered Jun 12 '22 at 03:03

Aspect

37
3

Shouldn't it be at least possible to compile the `-sys` crate's C code to wasm and link that? I'm unsurprised that it's not supported, but I don't think "cannot" is correct here. – Caesar Jun 12 '22 at 06:38
Thanks @Aspect ... I will avoid this kind of libraries – MrElephant Jun 12 '22 at 18:39

Using Wasm with data-fusion

1 Answers1