3

I have a very large tar file(>1GB) that needs to be checked out and is a precondition for executing any tests. I cannot have dedicated build server for my tests since tests are going to be executed on slave machines which are disposable. Checking out a file(>1GB) is not optimal since in this case test execution time would increase because of precondition.What is the best optimal way of solving this problem?

Jens Schauder
  • 77,657
  • 34
  • 181
  • 348
Mentor S
  • 41
  • 1

1 Answers1

1

I would dedicate a location on the slaves for that file.

Then in your tests, check if the file is in that location. If not, check it out and move it there. Since this location is outside your normal work area it won't get cleaned, and the file will stay there for the next test execution to use, and you won't need to check it out again.

Of course if the file changes you have to clear those caches. A first option would be to do this manual, alternative you can create a hash of the file and keep that hash in the cash and in your version control. You would then compare only the hashes, and only if those change you would check out the file.

Of course this requires that you have the ability to checkout all the rest of your code without the big file. How to do that obviously depends on the version control system in use.

Jens Schauder
  • 77,657
  • 34
  • 181
  • 348
  • Thanks Jens!.I was googling around and found few answers without explanation mentioning that image of large file should be created .However what is the advantage of that ,I am not clear.Does this help in performance ?. – Mentor S Dec 12 '15 at 19:43
  • If the large file doesn't change for many test runs, you only load it once (per build slave), so yes this might improve performance, when the number of test before the file changes is much greater then the number of build slaves. – Jens Schauder Dec 13 '15 at 07:48
  • I am not clear on how to create an image and also load it once per build slave.Can you point me to a link that can explain it. – Mentor S Dec 13 '15 at 15:04
  • Please don't ask new questions in the comments, just create a new question for it. – Jens Schauder Dec 13 '15 at 15:10