0

We are running a Jenkins server with multiple nodes. After some time and more and more projects that got added to the instance, we are encountering multiple issues (that we have to few people to solve, but that's another topic).

One really concerning is the size of the VM we use as a server. It exceeds 10 TB of data stored, and it will only grow (We need to access artifacts years/decades later).

The question for me at this point is generally if anyone has some experience with these large installations and can give me some estimates, of what might happen to the hole system if not taken proper care of and what recommended measures we should take from here.

Where we want to go from here is unclear. I just want some opinions on what more experienced people would do.

Marvin S.
  • 1

1 Answers1

0

Capacity planning also includes telling the organization how much their current way of doing things costs.

Estimate how much space you need for the next year or so. Round up a bit, planning to be 75% full gives a little buffer. Add costs you expect from not only the usable capacity, but exceeding the maximum size of one LUN/file system/storage array. Plus costs for redundant storage and backup.

Provide alternatives that cost less.

"Discard old builds" is a Jenkins option that controls retention. The actual requirements may come out when the price tag is known. Although, many organizations don't directly pay IT bills, so their business unit might not care.

Discuss how far in advance old artifacts can be requested. Getting it in a few days means it could be on a cold storage tier. Backup tapes can be a lot cheaper than an online system. And a separate backup system with different media improves the odds of the data surviving "decades". Find out how many decades, having hardware and people available to maintain forty year old data can get expensive.

Find out how reproducible the outputs are. If all the inputs are in version control, and the output is consistent, the outputs may not need to be archived for as long. Although, this requires disciplined procedures. Building software in particular is not easy to make outputs deterministic.

John Mahowald
  • 32,050
  • 2
  • 19
  • 34