Bootstrapping a Staging Clojure App via REPL incl. fetching dependenices

Question

Deploying Clojure/Java apps is hard, so I had this idea yesterday I want to understand better. If I spin up a machine that has Clojure and boot-clj installed and run boot wait repl -s -H 0.0.0.0 on the machine (let's ignore auth for now), I should be able to connect to it from my dev box and trigger the retrieval of dependencies over the wire (which will then be cached on the machine), then wire over all the source code and eval until I hit a snag, right?

Let's pretend this is a good idea. Is it possible to do this, and what are the hurdles involved? Right now I'm waiting 5 minutes for CircleCI to package up an uberjar, then fail because some Heroku token expired but all I want to do is see my code running on a staging environment so I can wire some more code and re-eval it.

The first thing t hat comes to mind is nREPL auth, which I see is not mentioned in any of the nREPL libraries. So let's say that's a higher-level networking concern and I'll do ACL via VPC.

Has anyone done this? Why is it a bad idea? Can you show your recipe for bootstrapping a Clojure app on a remote machine without the use of git or SSH (aside from initial REPL start)?

Answer 1

I should be able to connect to it from my dev box and trigger the retrieval of dependencies over the wire (which will then be cached on the machine), then wire over all the source code and eval until I hit a snag, right?

Is it possible to do this, and what are the hurdles involved?

Yes, but you need to specify -b (address server listens on) instead of -H (host to connect client to):

$ boot wait repl -s -b 0.0.0.0 -p 3000
nREPL server started on port 3000 on host 0.0.0.0 - nrepl://0.0.0.0:3000

Then connect to it however you like, for example with lein repl:

$ lein repl :connect 127.0.0.1:3000

Now you can add a dependency in the REPL and it'll be downloaded on the server/host. In the client REPL:

boot.user=> (set-env! :dependencies #(into % '[[clj-time "0.14.0"]]))

And if you're watching the server console you'll see it downloading dependencies:

Retrieving clj-time-0.14.0.pom from https://repo.clojars.org/ (3k)
Retrieving joda-time-2.9.7.pom from https://repo1.maven.org/maven2/ (32k)
Retrieving clj-time-0.14.0.jar from https://repo.clojars.org/ (22k)
Retrieving joda-time-2.9.7.jar from https://repo1.maven.org/maven2/ (618k)

And then back on the client side:

boot.user=> (require '[clj-time.core :refer [now]])
nil
boot.user=> (now)
#object[org.joda.time.DateTime 0x1f68b743 "2018-03-15T12:16:29.342Z"]

Has anyone done this?

Yes, I've seen people host nREPLs from remote servers and connect to them to tinker with a running system.

Why is it a bad idea?

Generally speaking, we want reproducible builds and stable artifacts to give some degree of certainty about what code is being released. Doing this type of development on-the-fly on-the-server works against those goals, making it harder to determine what code is running where. I'd try to structure the system (and its testing) such that this degree of remote dynamism isn't required for normal development.

It sounds like your primary problem is a cumbersome link (CI/CD) in your dev/test/run feedback loop. I'd explore other options for optimizing that feedback loop before going to dynamic dependency-hot-loading nREPL, if you can avoid it. Of course, it's there if you need it!

Can you show your recipe for bootstrapping a Clojure app on a remote machine

Personally, I only ever deploy JARs to remote machines, and usually in a container. By that time I've already exercised/tested the system locally and have some confidence it'll behave as expected. If most of your system is untestable without deploying, that may be a sign you should break it into smaller, more testable pieces.