3

Say I'm building an app in Racket.

And say eventually I want to compile that app as a single binary file that could be distributed to users, without them having Racket or any other software libs installed. I believe this is possible, yes?

Say in that app I want to use the snappy package https://docs.racket-lang.org/snappy/ which is some FFI wrappers around a C++ lib.

I already ran into a minor problem. I did (require snappy) inside DrRacket and followed the prompts and got the package installed but I get the error:

../../Applications/Racket v7.7/collects/racket/private/kw.rkt:1349:57:
ffi-lib: couldn't open "libsnappy.1.dylib" (dlopen(libsnappy.1.dylib, 6): image not found)

I can assume from this that racket-snappy expects the files for libsnappy to be on the usual unix path, but I'm on macos and mine is installed via Homebrew somewhere else. I believe the answer to that problem is here https://stackoverflow.com/a/24287418/202168

My concern is: I do not want users of my app to have to install these libs via Homebrew and fiddle with paths etc.

I am a Racket noob and know basically nothing about the compiler toolchain or C/C++ for that matter either. But I believe what I need is when I compile my Racket project to be able to have raco exe(?) "statically link" the libsnappy that's on my system and roll everything into a single binary with no dependencies.

So my question is: is this possible? If so, is it straightforward (i.e. managed via raco tools)?

I'm imagining in the worst case I have to download all the dependencies and build them from source and build my Racket project also as a library and then have some kind of skeleton C project that pulls them all in into one thing. I hope not.

I will add also... if this is easier in other Schemes (Chicken? Chez? Gambit? Guile?) then I'd be interested to know too.

Update: I found this article with some year-old anecdata of someone attempting the same thing https://taoofmac.com/space/blog/2019/06/20/2310

Based on that, and Ryan's answer below, raco distribute looks promising and I really need to try this out for myself to confirm what works.

Update again: Here is another article again confirming raco distribute should put everything into a folder with no external deps https://defn.io/2020/06/28/racket-deployment/ and here is a pointer to the docs for how to build a .dmg image for MacOS: https://docs.racket-lang.org/raco/exe-dist.html#(part._.A.P.I_for_.Bundling_.Distributions)

Anentropic
  • 32,188
  • 12
  • 99
  • 147
  • I found this for Chez https://github.com/gwatt/chez-exe ...can anyone confirm if it does what I'm asking for? It sounds like it might (but only for Chez Scheme obviously) – Anentropic Jul 19 '20 at 23:18
  • there appears to be a way for Chicken Scheme described here: http://www.foldling.org/scheme.html#2013-07-16 – Anentropic Jul 19 '20 at 23:27
  • it seems `chez-exe` does not do what I want https://github.com/gwatt/chez-exe/issues/2 – Anentropic Jul 19 '20 at 23:36
  • 2
    `raco distribute` makes a version of an executable (that you made with `raco exe`) which sits in a directory structure where any *racket* shared libraries are next to it. For third-party shared libraries I think you'll need to put them somewhere sensible and adjust paths suitably using a `config.rktd` during the build. If you really want static binaries I have no idea, but I suspect it's not possible at all on OSX for anything. –  Jul 20 '20 at 11:49
  • I'm really looking for a solution to 3rd party shared libraries, the kind of C or C++ libs where the Racket part is mostly just FFI bindings.... is it possible to specify the path to the `.dylib` or `.so` file as a relative path and make them part of the distributed package? – Anentropic Jul 20 '20 at 12:48
  • Shared libraries specified via relative path, where I can put all the dependencies in a single package directory would work for me... after all on MacOS what looks like an application file is really a directory. What I need to avoid is having any hard-coded absolute paths or asking the user to configure paths in order to use the app – Anentropic Jul 20 '20 at 12:53
  • 1
    For Racket's own libraries, yes, I think so (one problem is that for the only thing I build that way, `raco exe` now builds a binary which doesn't need any Racket libraries at runtime so I can no longer tell). For third-party shared libraries I'm not sure but I would expect so if you are careful. –  Jul 20 '20 at 13:13

1 Answers1

3

There is a partial solution using a combination of raco distribute and define-runtime-path.

Suppose you have a program that uses libzmq, which you know is installed on your build system at /usr/lib/x86_64-linux-gnu/libzmq.so.5. You can use define-runtime-path to create a reference to that file and tell raco distribute to copy it to the distribution directory. For example, suppose that "my-app.rkt" is the following:

#lang racket/base
(require racket/runtime-path)
(define-runtime-path zmq "/usr/lib/x86_64-linux-gnu/libzmq.so.5")
(printf "zmq = ~e\n" zmq)

When you run the program using racket my-app.rkt, it prints

zmq = <path:/usr/lib/x86_64-linux-gnu/libzmq.so.5>

But when you run raco exe my-app.rkt and then raco distribute MyApp my-app, then the MyApp directory will contain a copy of libzmq.so.5:

$ find MyApp/ -type f 
MyApp/lib/plt/my-app/exts/ert/r0/libzmq.so.5
MyApp/lib/plt/racket3m-7.7
MyApp/bin/my-app

and if you run ./MyApp/bin/my-app, it prints

zmq = #<path:/PATH/TO/HERE/./MyApp/bin/../lib/plt/my-app/exts/ert/r0/libzmq.so.5>

You can use (ffi-lib zmq) to load the shared library. Unfortunately, that directory is not in the search path that the application will use for loading shared libraries, so existing Racket libraries that just try to load (ffi-lib "libzmq" '("5")) won't find the application's copy.

There is another way of using define-runtime-path specifically for shared libraries, and I thought it would solve that problem, but it doesn't seem to. That seems like a bug to me, so I'll file a bug report.

Update: I have filed a bug report about the fact that define-runtime-path's shared library ('so) mode causes raco distribute to copy the shared library outside of the application's library search path.

Ryan Culpepper
  • 10,495
  • 4
  • 31
  • 30
  • 1
    Thank you for your answer, this looks like the info I needed. Do I understand correctly the last paragraphs: if I write my own Racket FFI binding to a C lib things will work (because `define-runtime-path` is in my project) but if I use 3rd party Racket bindings to a C lib then it might not? – Anentropic Jul 21 '20 at 08:54
  • 1
    Right. Specifically, your own FFI binding will work if you use the variable defined by `define-runtime-path` to load the foreign library. 3rd party FFI bindings probably won't work because they would be using the usual Racket+OS library search path. – Ryan Culpepper Jul 21 '20 at 12:10
  • What about if I manually symlink the library files into the path expected by the 3rd party bindings on my machine... then I wouldn't need `define-runtime-path`... should `raco distribute` then successfully copy them into the dist folder? – Anentropic Jul 21 '20 at 13:18
  • No, because `raco distribute` would not be aware of the dependency, because `ffi-lib` does not register a dependency like `define-runtime-path` does. And in general it can't, because its arguments can be computed dynamically, and because the code might have several calls to `ffi-lib` but only want to use the first one that succeeds, and so on. – Ryan Culpepper Jul 21 '20 at 18:33
  • Hmm, it seems like I have to audit every 3rd party lib that provides FFI bindings to know whether they are compatible with `raco distribute` then? If they use `define-runtime-path` it should be ok, if they have hard-coded paths then my distribution will not be self-contained? – Anentropic Jul 21 '20 at 18:40
  • Here's another idea: I think you could work around the problem by manually copying the shared libraries into the appropriate subdirectory directory created by `raco distribute`. I haven't tried it, and it would be brittle and awkward to automate, but it should work. In Racket 7.7, based on my one experiment, the right subdirectory seems to be `lib/plt//lib`. If you manually copy shared libraries to that directory, the application (and any 3rd party Racket libraries it uses) should find them. – Ryan Culpepper Jul 21 '20 at 18:43
  • Thanks so much for your help. I updated the question with a couple of articles I found, it looks like `raco distribute` should generally cope with that, e.g. https://defn.io/2020/06/28/racket-deployment/ shows it successfully adding copies of `libssl.1.1.dylib` and `libcrypto.1.1.dylib` to the `dist/lib/plt/...` folder. I think I have to try it myself and see. My question was to research in advance in case it was totally impossible, but I feel like Racket will probably be fine for this I think I will like this language... – Anentropic Jul 21 '20 at 18:50