0

I'll be using Incredibuild or Electric-Cloud, both are similar to what distcc does.

CONFIG

  • two PCs
  • inside my company (so no slowdown due to firewall issue)
  • both on the same (really) expensive gigabit switch

I know NOTHING about networking but . . . my concern is that because my .cpp files are so small - 96% of ~1800 < 100KB - my distributed build tool is gonna waste a lot of time transferring the .cpp/artifacts in an inefficient way.

Is there a difference in transfer speed between

bursty, on-demand transfers aka for each .cpp/.obj/artifact

vs

transferring the inputs and artifacts "in bulk"?

Bob
  • 4,576
  • 7
  • 39
  • 107

1 Answers1

0

If you are using IncrediBuild you shouldn't concern yourself with these kind of issues as IncrediBuild manages everything transparently for you. You don't need to explicitly pass files or set up anything else on the remote machines besides a very light IncrediBuild Agent. IncrediBuild's process level virtualization makes sure that once a process is being executed on the remote machine, only the actual files that the process requires in order to work properly are synchronized to the remote machine (this is all done by IncrediBuild in the background and you don't need to do anything yourself but executing you regular build command). Regarding the files sizes and the effect on the network, IncrediBuild has various mechanisms that minimize network traffic such as run time compression of files being transferred, a smart caching mechanism on the remote nodes (which makes sure that files won't need to be transferred again if they were transferred before – even if it was from a different initiating machine). Bottom line, the impact on the network is minimal, and after the first couple of builds (once the cache is being populated on the remote machines), the impact on the performance and network is insignificant. IncrediBuild is being used by customers with code bases containing gigs of source files without any problem. You are welcomed to contact support@IncrediBuild.com for further information.

Disclaimer, the writer of this answer works at IncrediBuild.

Dori
  • 470
  • 2
  • 6