4

I want to know the recommended way to use shake and ghc --make together. In my project, the shake rules are used to compile C source code into their *.o files (which I call cobjects), and these objects are linked together into my haskell program by calling ghc --make. During this command, ghc figures out for itself when it needs to rebuild my haskell files.

The example below isn't correctly written to integrate the two tools. If a .hs file changes then rerunning the shake script won't re-invoke ghc --make. I understand why shake doesn't know to rebuild, but I don't know the recommended fix.

main = shake shakeOptions $ do
  want [main_exe]

  main_exe *> \out -> do
    need cobjects
    cmd "ghc --make" hs_compileFlags cobjects "main.hs"

  cobjects **> ...
Ein
  • 1,553
  • 3
  • 15
  • 22
  • Question: when you say "when a .hs file changes" - do you mean "when `main.hs` changes`? or are there other .hs files involved? It seems to me that your rules for `main_exe` should have `needs "main.hs"` in it somewhere. – ErikR Feb 13 '14 at 18:50
  • No, I mean "when any .hs file changes". If `main.hs` is unchanged, but another source file `foo.hs` that `main.hs` depends on has changed, then the command `ghc --make` is smart enough to figure out by itself that it must recompile `main.hs` and whatever else depended on `foo.hs`. – Ein Feb 13 '14 at 19:34
  • 1
    Part of the answer might be in using `ghc -M` to have ghc emit dependency information -- see [this](http://www.haskell.org/ghc/docs/latest/html/users_guide/separate-compilation.html#makefile-dependencies) for more details. You could then incorporate that into a `needs` statement. – ErikR Feb 13 '14 at 20:03
  • 1
    Instead of re-running `ghc --make` "when any .hs file changes", don't you want to always re-run `ghc --make`? That seems necessary to re-link the executable, and then `ghc --make` should pick up any changes to .hs files as well. – John L Feb 13 '14 at 23:09
  • Yes, I always want to run `ghc --make`. It should run after (possibly) doing the rule for `cobjects` – Ein Feb 14 '14 at 01:16
  • @JohnL to run something "every time" you could do it with `action $ need [main_exe]; cmd "ghc --make" ...`, but it's not a good solution as Shake is no longer doing proper dependencies for anything you add downstream of `main_exe`. In addition, running `ghc --make` is actually quite slow (I have 0.003s vs 10s for some projects as `ghc --make` invokes the C pre processor and parses everything). Fortunately using `ghc -M` works well. – Neil Mitchell Feb 18 '14 at 14:10

2 Answers2

2

The general principle is that if a rule runs a command that uses a file, it should depend on the file. In your case you must need all the things ghc --make requests, and the easiest way to figure that out is with ghc -M (which generates a Makefile):

main = shake shakeOptions $ do
  want [main_exe]

  main_exe *> \out -> do
    need cobjects
    cmd "ghc -M" hs_compileFlags "main.hs"
    cmd "ghc --make" hs_compileFlags cobjects "main.hs"
    needMakefileDependencies "Makefile"

  cobjects **> ...

The needMakefileDependencies function is in Development.Shake.Util. The only subtlety is that I call needMakefileDependencies after running the command, since otherwise the object files won't have been created yet (you could solve that by filtering only the .hs files in the Makefile using parseMakefile yourself).

Neil Mitchell
  • 9,090
  • 1
  • 27
  • 85
  • 1
    It's not quite so simple as just `need`ing everything `ghc -M` requests. You must also arrange to call `ghc` on every file that might need recompilation, which includes changes via `CPP`, changed compilation flags, template haskell's `addDependentFile` (`ghc -M` doesn't track this), and any time the package db has changed. Also there are a few odd errors having to do with stale `.hi/.o` files if a source file has been deleted. These may not be relevant to every project, but broken build systems are supremely frustrating so it's good to know about them. – John L Feb 18 '14 at 23:24
  • As @JohnL says, `addDependentFile` won't work, and you might want to call `ghc --make` if the output of `ghc-pkg list` changes, which captures any package change. I don't think you need to call `ghc` on every file, since the `--make` mode will take care of that. There shouldn't be any errors with stale `.hi/.o` files, even if things have been deleted (other than ones stemming from `ghc --make` alone). – Neil Mitchell Feb 23 '14 at 22:23
  • I agree, calling `ghc --make` should take care of all issues related to those problems. I meant if you want to entirely avoid `ghc --make`, those are issues that you will need to manage yourself. – John L Feb 23 '14 at 23:58
0

It looks like recommended way might be to use "phony" rule and add a want for that rule at the top. This was found from this helpful tutorial:

http://blog.jle.im/entry/shake-task-automation-and-scripting-in-haskell

My first attempt at adapting this idea to my example:

main = shake shakeOptions $ do
  want [main_exe, "compile and link hs"]

  main_exe *> \out -> do
    need cobjects

  cobjects **> ...

  "compile and link hs" ~> do
    cmd "ghc --make" hs_compileFlags cobjects "main.hs"

The (~>) function specifies that the string "compile and link hs" is the name of a rule but not the name of a file.

While this will always attempt to run the new rule like I want, it unfortunately tries to run the rule before the rule for main_exe which isn't what I want.

Moving the demand for the new rule inside the rule for main_exe seems to fix this new problem:

main = shake shakeOptions $ do
  want [main_exe]

  main_exe *> \out -> do
    need cobjects
    need ["compile and link hs"]

  cobjects **> ...

  "compile and link hs" ~> do
    cmd "ghc --make" hs_compileFlags cobjects "main.hs"

Although this works I don't understand how Shake decides which order to run the two rules in the first piece of code.

Ein
  • 1,553
  • 3
  • 15
  • 22
  • In general it isn't a good idea to `need` a phony rule from a non-phony rule. I suspect if you replaced the `need ["compile and link hs"]` with the `cmd "ghc --make" ...` it would do exactly the same, which gets you back to what you had in the question. – Neil Mitchell Feb 18 '14 at 14:07