3

I'm confused about the rules Shake uses to work out whether an output needs to be rebuilt. I have a simple build for documents with two steps. The full build file is below but to summarise, asciidoc is used to transform a .txt file into a .dbxml (Docbook XML) file, which is then transformed into PDF using dblatex.

I expect that if I touch the PDF and rerun shake, nothing should happen because the output is newer than both inputs. However, shake in fact executes the dblatex step.

Next, I expect that if I touch the .dbxml file then shake will execute dblatex but not asciidoc, because the dbxml is newer than its input (i.e. the .txt). However shake in fact executes both asciidoc and dblatex steps.

Have I made a mistake in my dependencies?

import Development.Shake
import Development.Shake.FilePath

-- List of output files
outputs = ["process.pdf"]

main = shakeArgs shakeOptions{shakeVerbosity=Diagnostic} $ do
    want outputs

    -- Rule to produce pdf files from dbxml inputs
    "*.pdf" *> \out -> do
        let dbxml = out `replaceExtension` "dbxml"
        need [dbxml]
        cmd "dblatex" "-o" out dbxml

    -- Rule to produce dbxml files from txt (asciidoc) inputs
    "*.dbxml" *> \out -> do
        let src = out `replaceExtension` "txt"
        need [src]
        cmd "asciidoc" "--backend=docbook45" "--doctype=article" "-o" out src
Neil Bartlett
  • 23,743
  • 4
  • 44
  • 77

1 Answers1

4

In Shake a file is considered dirty if its last modified time changes from when it was built. In make a file is considered dirty if its last modified time is older than its dependencies. I suspect your observations all stem from this difference. To directly answer the question, Shake rebuilds a file if it or any of its direct dependencies have changed.

Why does Shake do something different to make? Three reasons:

  1. Make doesn't store enough information to allow it to rebuild if the modification time changes since it doesn't record any extra metadata about what the modification time was in the last build.
  2. By doing time comparison make relies on a monotonically increasing clock, which breaks if the user changes their system time, and seems particularly prone to breakage on NFS filesystems.
  3. Assume A depends on B, and B depends on C. C changes, and B rebuilds, but the rule for B is clever enough to spot that the old B doesn't need updating. In make you have two bad options - touch B anyway and rebuild A, or don't touch B and then run the rule for B every time you run make. In Shake you don't touch B, A doesn't rebuild, and B doesn't rebuild next time.

As for your build system, it all looks good to me. My only minor tweak would be the use of the infix operator -<.> instead of replaceExtension - they are both the same function but the operator looks clearer to my eye.

Neil Mitchell
  • 9,090
  • 1
  • 27
  • 85
  • Thanks for the explanation. Unfortunately it means I probably have to redesign my build system. I wanted to be able to make some manual changes to the intermediate dbxml files, and not have them overwritten by the asciidoc generator. – Neil Bartlett Oct 21 '13 at 19:47
  • Making manual changes to generated files is usually a poor plan, since if you ever change the underlying source files your changes will get lost. Perhaps drop an email to the [Shake mailing list](https://groups.google.com/forum/?fromgroups#!forum/shake-build-system) describing the workflow you are hoping for? – Neil Mitchell Oct 22 '13 at 09:07
  • Yes I agree it's probably dangerous. I should move the dbxml into a source folder when I want to make this change. So I think I want a switch in the PDF rule that generates from the source folder dbxml file if it exists, otherwise it first builds the dbxml using asciidoc. – Neil Bartlett Oct 22 '13 at 16:01
  • Sounds like a plan. Something like: `src <- doesFileExist "foo.dbxml"; let dbxml = if src then ["foo.dbxml"] else ["_make/foo.dbxml"]; need dbxml` - and even the `doesFileExist` is tracked! – Neil Mitchell Oct 22 '13 at 17:28