1

Given a non-homogeneously structured mixed-language code base, what is the recommended way to specify metadata to drive a Shake build system?

In particular, the metadata should describe source language (C++, C#, Fortran), source files, result type (static/dynamic lib, executable), compiler switches (potentially different for each artifact), etc.

Preferably, the metadata should be simply structured and stored in one separate file per artifact.

Is there a smart way to generalize the approach suggested in Defining your own build system with Shake?

precarious
  • 598
  • 2
  • 14

1 Answers1

1

The approach from the presentation scales quite far - I've used it for huge multilanguage projects. I've used three tweaks beyond the talk:

Add file extensions

Typically file extensions give you the source language and result type. For example:

mycsharp.dll = foo.cs bar.cs
myfortran.exe = main.f90 util.f90
docs.pdf = docs.tex references.bib

Now you can have entirely different rules to interpret Fortran executables, C# executables (or dlls) and PDF documents.

Add some 'leading' characters

Often you want data about flags, or other command-line relevant data. For example:

mycsharp.dll = foo.cs bar.cs -define:DEBUG -optimize +mono

I tend to use special leading characters. In the above example I've used - to denote flags (which are usually passed on verbatim), and + to denote a selection from an enumeration which contains useful special cases (e.g. use the mono compiler).

One word of caution, don't use too many weird leading special characters, or you end up inventing your own language - keep it simple.

Use a C pre processor (CPP)

The C pre processor gives you #include, #define and #ifdef, all of which can be used in more complex structured metadata. You can use this with Shake by invoking cpphs on the metadata file first.

While the two previous tweaks are recommended, the use of CPP was originally for #include. Now the built-in Shake metadata has an include mechanism I'm not sure I'd bother with CPP, which keeps things simpler.

Neil Mitchell
  • 9,090
  • 1
  • 27
  • 85
  • Encoding metadata into variable names and using special characters to distinguish types of information are nice tweaks. Thanks! – precarious Nov 19 '15 at 20:26
  • Another technique that came to my mind is generating a Shake config file from custom config files distributed in the file system. In order to establish a better build system at my work place it would be helpful, I think, if at least during a transition phase these custom config files would be Visual Studio project files. Currently, the build is driven by some Python scripts and a bunch of Visual Studio solutions that contain about 500 projects. Do you think the general approach is feasible? I understand that it involves a non-negligible amount of work. – precarious Nov 19 '15 at 20:29
  • @precarious: That's exactly what we did it my work, but driven from Makefiles (since that was our original format) - see http://neilmitchell.blogspot.co.uk/2014/07/converting-make-to-shake.html. Originally we had a script that translated Makefile to custom configs which ran at the start of every build (just a Haskell script before even getting to Shake). To transition, we deleted the Makefiles and checked in the generated configs. It wasn't that much work as the parser/translator could be incredibly hacky and temporary, with special cases for anything weird. – Neil Mitchell Nov 20 '15 at 09:23