4

I manage a build server for an open source project and it often needs to rebuild huge libraries that take at least 30 minutes to build on a 8-logical-core i7 Intel processor, maxing at several hours.

Lately I was wondering if there are some relatively cheap dedicated multicore boards which I could use for speeding up compilation times, but my google-fu couldn't find anything except Parallella on Kickstarter, googling which doesn't seem to come up with many results of my use case.

Do you know of any cheap multicore boards that I could use to speed up compilation of C/C++ software?

Cheap on-demand could services providing multicore/distributed compilation would work, but they are a lot less favorable since you would need to check that their compilers, libraries and whatnot are authentic, so that they won't be injecting any backdoors in whatever code you compile, which is quite hard to check. That's why I'm primarily looking for a multicore boards that I could manage myself to offload the compiling task to.

Vanilla Gorilla
  • 269
  • 2
  • 13
  • 3
    I'd look into other ways to speed it up first (which should be better too). Make sure that the compilation is really running parallel or your cores won't do anything. Make sure that you use incremental builds so you don't have to build everything from scratch all the time, etc... – AliciaBytes May 24 '15 at 20:26
  • Compilation does run in parallel. Incremental builds it's something to investigate, we currently download tarballs of new source releases of libraries we use, instead of pulling the updates from git, because doing so takes a lot more space and time, even for shallow clones. – Vanilla Gorilla May 24 '15 at 20:33
  • In addition to @RaphaelMiedl comment .. if that's Linux server make sure ccache is installed and the projects being built use it – doqtor May 24 '15 at 20:33
  • Are you using make? (or something similar). You could use the "-j8` option in order to perform 8 make actions in parallel – lrleon May 24 '15 at 20:41
  • @lrleon As I said, the compilation is already done in parallel and I'm looking for a dedicated board or something of that kind, not for a software solution. – Vanilla Gorilla May 24 '15 at 20:43
  • Did you know about distcc? https://code.google.com/p/distcc/ – davlet May 24 '15 at 20:45
  • @davlet I have it in tags... – Vanilla Gorilla May 24 '15 at 20:47
  • Might be worth taking a look at [Tup](http://gittup.org/tup/) which purports to be much faster than *make* and calculates dependencies automagically. [Make vs Tup](http://gittup.org/tup/make_vs_tup.html) – Galik May 24 '15 at 21:13
  • In one project we've reduced the compilation time from 2 hours to 40 minutes by optimizing make. Make used to perform too much checks for selecting the tool to use (e.g. location of cc). The results for those checks were not saved, so they were performed again and again. Try to replace `make` by `strace make` to see if it's still the case. Sorry it's a software solution again, but simply replacing make-tool shouldn't make much side-effects. – Valentin H May 24 '15 at 21:15
  • @Galik Funny coincidence. Similar suggestion at almost the same time :-) Good to know, that there is a sth like Tup! – Valentin H May 24 '15 at 21:18

0 Answers0