shake build: how to deal with case where the needed file are discovered later

Question

I use shake to build a website (with pandoc). When files are converted to pandoc, other files (css, bibliography, templates, etc.) may be needed, but shake does not know it, because it is internal to the pandoc calling function and the information is in the files used and only gradually visible.

I have the impression from reading the docs, that asking the function called by shake to return a list of the files used and, after the function called in shake, to use the returned list of files to call need. Is it correct that the order in which need is called, matters?

Alternatively,

(1) I can build functions to only find which other files are needed (doing the work nearly twice) and call them first. Or,

(2), to break the process into steps, each resulting in a file and then start a new rule to go forward from this file (and the additional files) and add needs there. The second solution builds intermediate files and breaks the logical flow of the transformation from pandoc to html.

What is better?

Answer 1

The answer depends on the details of the files that are depended upon (e.g. the css, bibliography):

• If they are source files, you can add the dependency after they are used, using needed.
• If they are generated by Shake itself, you can't use needed without first ensuring they are present. If you can statically predict a superset of which files are the dependencies consulted, you can use orderOnly to ensure all the files that the rule might depend on have been built, then use needed after to declare which are actually required.
• Finally, if you can't predict which files are dependencies, and you might be generating them, then the easiest is to run part of the computation twice.