1

I have been using Clojure, ClojureScript, lein, shadow-cljs, re-frame, reagent, Emacs, and CIDER to work on a Clojure/ClojureScript dynamic web app project.

Usually, I build the project by executing shadow-cljs watch app. It works fine. I can use the application and watch changes.

Currently, I am working on a Continuous Integration project via GitHub Actions. In this new environment, I wanna use a different command: shadow-cljs compile app.

But, I am having problems. Both on GitHub Actions env and in my local env (which is reproducing the steps on GitHub Actions).

The result returned by the command is quite weird. First, it says the compilation went well, Build completed, and the terminal displays some non-harmful warnings:

shadow-cljs compile app
shadow-cljs - config: /Users/pedro/projects/my_project/shadow-cljs.edn
WARNING: random-uuid already refers to: #'clojure.core/random-uuid in namespace: portal.runtime.browser, being replaced by: #'portal.runtime.browser/random-uuid
[:app] Compiling ...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
WARNING: abs already refers to: #'clojure.core/abs in namespace: day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color, being replaced by: #'day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color/abs
WARNING: abs already refers to: #'clojure.core/abs in namespace: garden.color, being replaced by: #'garden.color/abs
[:app] Build completed. (1121 files, 27 compiled, 8 warnings, 45.85s)

------ WARNING #1 - :redef -----------------------------------------------------
(... omitted ...)

Unfortunately, the terminal stays like this forever. It behaves like the watch command... But, compile is not supposed to be like this.

The terminal does not finish the process. There is no exit. Even though the build is complete, as stated in the last line before WARNING.

Feels like a deadlock situation. The terminal is still responsive because I can cancel the process with Control-c. But, besides canceling it, it is frozen. This can be catastrophic on GitHub actions since you are going to be paying for an ongoing process.

On shadow-clj.edn, I tried commenting out compilation-options:

;:compiler-options {:optimizations :none}

But it did not make any difference. The same problem happens with shadow-cljs release app.

After finding a GitHub issue with a similar problem, I also tried:

:js-options {:resolve {"highlight.js" {:target :npm :require "highlight.js/lib/core"}}}

One of my hypotheses is that highlight.js is causing some trouble while parsing strings and/or files...

I am also afraid the problem could be some REPL being fired up by some dependency and blocking the process for the exit... But I am not sure where to look for it.

Why is this happening? What could be causing this? How can I solve it?

;;;;

UPDATE:

After @thomasheller kind answer, I tried some things. I believe all of the hints can be discarded for the present situation, except for the macros:

1 - The project does not use user.clj

2 - Build hooks is the default setting which is basically a comment:

:build-hooks [;; this will create a build report for every release build
                                    ;; which includes a detailed breakdown of the included sources
                                    ;; and how much they each contributed to the overall size:
                                    #_(shadow.cljs.build-report/hook
                                       {:output-to "build-reports/report.html"})]

3- There is no shadow-cljs start, server, or watch process going before trying to compile. The problem of an endless compile app happens just after re-starting my MacBook, on the CI, in others' people's PC... So, it is unrelated.

Ok. Now, let's talk about the macros... The project's main repository has 6 defmacros - according to a git grep (I have not checked the dependencies).

I am suspicious about one of the macros which is involved in reading files:

(defmacro slurp [file & [default]]
   (if (.exists (io/file file))
     (clojure.core/slurp file)
     default))

Why does this exist? Well, it is being used inside another macro that reads config files related to environment variables:

(defmacro read-open-config [env-var]

  (clojure.edn/read-string (slurp (str "config/" (System/getenv env-var) "-open.edn"))))

We have multiple config files for different purposes (I know it is not the standard practice...).

It feels like too much dynamicity...

Pedro Delfino
  • 2,421
  • 1
  • 15
  • 30

2 Answers2

3

First of all a few clarifications.

  • compile produces a development build, no optimizations are applied. Setting anything related to that does nothing.
  • CI systems should likely use release. This avoid including all the development related code (eg. highlight.js, re-frame-10x).
  • A dependency cannot "fire up a REPL". The shadow-cljs "server mode" provides it, which compile or release don't normally enter.
  • highlight.js errors would fail earlier with a visible error, you would not get to "Build completed."

This could be happening for a variety of reasons, very hard to debug without seeing the build config/setup. A few guesses:

  • You have a user.clj on the classpath, which starts additional "stuff" when the process is launched and this namespace is loaded? Clojure will load this unconditionally and shadow-cljs cannot prevent whatever that may do.
  • You have configured :build-hooks that launches additional stuff. It could launch an addition css watch process that doesn't exit?
  • You use a macro that launches additional stuff without shutting it down?
  • You previously started a shadow-cljs start, server or watch process? Coupled with the above that may then wait for the server to stop. On your local machine is the watch maybe still running? You can prevent the attempted server connect via npx shadow-cljs release build --force-spawn, but that is only useful if it is actually running.

Normally shadow-cljs will shut down after compile or release. I'm not aware of anyone ever having any issues with this after a successful build. The issue you linked is not related to yours, since that never got to a completed build.

Thomas Heller
  • 3,842
  • 1
  • 9
  • 9
  • Thanks again, @thomasheller. I have updated my question after doing some new research based on your suggestion. – Pedro Delfino Sep 30 '22 at 15:39
  • 1
    The macros are fine. They don't start processes or such things. I'm out of guesses. Can't say more without much more detail about your setup. – Thomas Heller Sep 30 '22 at 16:32
  • Ok. Thanks, again. How can I create a reproducible public example? what would be the minimum necessary information? – Pedro Delfino Sep 30 '22 at 16:49
  • 1
    If you can reproduce it with public code that would help a lot. Versions used, build config, exact commands used, etc is all relevant. Does that happen if you follow the [Quick Start](https://github.com/thheller/shadow-cljs#quick-start)? – Thomas Heller Sep 30 '22 at 20:25
0

After a long research process, it was discovered that the problem came from tests inside a private dependency. Apparently, some tests with asynchronous processes profile were holding the compilation process from finishing.

After commenting-out some olds tests, the compilation happened as expected.

Pedro Delfino
  • 2,421
  • 1
  • 15
  • 30