2

I'm looking for a storage system for replacing a custom built backup system. I have looked briefly at MogileFS and OpenStack Object Storage but I still don't know if any of these suits my needs since I need to archive disks offline when they're full.

Requirements:

  • Free.
  • Can handle 100M files up to a few MiB each.
  • Support offline/cold disks.
  • Supports multiple nodes at different locations.
  • Supports redundancy by storing files multiple times. E.g. "this file should be stored once in both datacenter X and Y. This file should be store only once and I don't care where."
  • Should use a custom API and not be POSIX filesystem compatible.
  • Should run in an Linux or Free BSD environment.

The hardest requirement is supporting offline/cold disks. This is due to that only a few disk can be online at a time. When they are full they will be replaced with empty disks and archived in a safe place.

Which software will fit my needs?

Martin Torhage
  • 161
  • 4

1 Answers1

3

I specifically asked a disk drive vendor if they support disks that are not powered on and stored in a specifically designed environment in a closed shelf etc. they answered that this is an unsupported mode of operation and that you cannot rely on powered-off disks to be ever usable again in case you need them.

There are special devices that power-off disks when not in use called MAID (massive array of idle disks) specifically designed to handle that.

In any case you'll need an enterprise grade object storage system that does support ILM/HSM to implement that, but they are not exactly "free".

pfo
  • 5,700
  • 24
  • 36
  • Good point. However, we have a very tight budget and we already own the servers and the 100+ disks that we're going to use so we basically have free hardware. Redundancy is based on multiple copies spread over multiple sites. That is good enough for us. – Martin Torhage Apr 11 '12 at 09:08