How to show progress in Shake?

Question

I am trying to figure out how can i take the progress info from a Progress type (in Development.Shake.Progress) to output it before executing a command. The possible desired output would be:

[1/9] Compiling src/Window/Window.cpp
[2/9] Compiling src/Window/GlfwError.cpp
[3/9] Compiling src/Window/GlfwContext.cpp
[4/9] Compiling src/Util/MemTrack.cpp
...

For now i am simulating this using some IORef that keeps the total (initially set to the sum of the source files) and a count that i increase before executing each build command, but this seems like a hackish solution to me.

On top of that this solution seems to work correctly on clean builds, but misbehaves on partial builds as the sum that displayed is still the total of all the source files.

With access to a Progress data type i will be able to calculate this fraction correctly using its countSkipped, countBuild, and countTodo members (see Progress.hs:53), but i am still not sure how i can i achieve this.

Any help is appreciated.

There does not seem to be much support for that. See https://groups.google.com/forum/#!topic/shake-build-system/ZPyTIfVL-dA for a recent discussion of a related proposal on the shake mailing list. — Joachim Breitner, Aug 21 '15 at 13:09

score 1 · Accepted Answer · answered Aug 21 '15 at 14:46

Values of type Progress are currently only available as an argument to the function stored in shakeProgress. You can obtain the Progress whenever you want with:

{-# LANGUAGE RecordWildCards #-}
import Development.Shake
import Data.IORef
import Data.Monoid
import Control.Monad

main = do
    ref <- newIORef $ return mempty
    shakeArgs shakeOptions{shakeProgress = writeIORef ref} $ do
         want ["test" ++ show i | i <- [1..5]]
         "test*" %> \out -> do
             Progress{..} <- liftIO $ join $ readIORef ref
             putNormal $
                "[" ++ show (countBuilt + countSkipped + 1) ++
                "/" ++ show (countBuilt + countSkipped + countTodo) ++
                "] " ++ out
             writeFile' out ""

Here we create an IORef to squirrel away the argument passed to shakeProgress, then retrieve it later when running the rules. Running the above code I see:

[1/5] test5
[2/5] test4
[3/5] test3
[4/5] test2
[5/5] test1

Running at a higher level of parallelism gives less precise results - initially there are only 3 items in todo (Shake increments countTodo as it finds items todo, and spawns items as soon as it knows about any of them), and there are often two rules running at the same index (there is no information about how many are in progress). Given knowledge of your specific rules, you could refine the output, e.g. storing an IORef you increment to ensure the index was monotonic.

The reason this code is somewhat convoluted is that the Progress information was intended to be used for asynchronous progress messages, although your approach seems perfectly valid. It may be worth introducing a getProgress :: Action Progress function for synchronous progress messages.

Fantastic solution! Still would be nice to have a syncronous version as you said to.. — Fr0stBit, Aug 21 '15 at 19:51
@TheArtist: I raised a ticket to add it - shouldn't be too hard: https://github.com/ndmitchell/shake/issues/293 — Neil Mitchell, Aug 21 '15 at 20:16
I'll remark that this approach doesn't work so well for more complicated build systems which have a more heavy Haskell component, because there may be a large number of rules which you don't want to print output for, but they'll show up in the count. (Also, countTodo isn't perfectly accurate.) — Edward Z. Yang, Dec 19 '15 at 19:59

How to show progress in Shake?

1 Answers1