2

I love the idea of using SSD EBS instance stores as L2ARC and ZIL for a zpool backed by EBS.

Going further (and into more dangerous territory), could I instead create a zpool mirror with the 2 instance stores:

zpool create vol1 mirror xvdb xvdc

and then use ZFS snapshotting/replication to keep a "warm"/eventually consistent spare on EBS?

  • I would be OK with losing a few seconds of data
  • I don't want to add the EBS as a hot spare, because that would limit the speed of the whole pool
Seamus Abshere
  • 673
  • 1
  • 6
  • 10
  • I wouldn't recommend. The cloud isn't really a good application for ZFS in this manner. – ewwhite Jun 27 '14 at 21:44
  • This is contradicting. Not getting into implementation details, how can a slow storage keep-up replicating a fast one? It can't. Use EBS with S3 snapshots. That's the single most important benefit of using an expensive AWS server, S3 snapshot capability. If you don't want that, just use a cheap VPS from elswhere. – sivann May 29 '15 at 08:20
  • EBS can slow in the sense that it has (relatively) high latency especially for small random reads, but it can have high throughput especially for batched sequential writes. So if you application is read-heavy and latency sensitive, then it may make sense. – Zekian Aug 23 '23 at 12:31

3 Answers3

2

I would not mix such different disks in a mirrored volumes. I would rather use frequent send/receive iterations to have consistent, point-in-time backups of the main volume.

shodanshok
  • 47,711
  • 7
  • 111
  • 180
2

I would suggest not to mirror the drives. Instead, create 2 zpools with one drive each, one with the ephemeral drive then another with the EBS drive. Create a dataset then zfs send at frequent intervals to the EBS zpool from the ephemeral zpool snapshots. You can easily grow the EBS drive and zpool while your zpool is online using aws cli to grow the EBS snapshot, fdisk, parted and zpool online -e (expand). With snapshots rotation you could save space. For example, keep only the last 24hrs snapshot - if you perform a snapshot and send/recv in 10 minutes interval you would keep at a minimum 144 snapshots in a day.

soyayix
  • 61
  • 3
1

Seems to me a perfectly acceptable setup if your spare writes to EBS (and you snapshot that), and you have some failover scenario, as obviously restoring your instance store cost time.

What you describe was actually the only setup we had before we had EBS. People survived for years doing exactly that.

Finally, Netflix moved away from EBS backed disks due to extra risk of failure. They just replicate using instance storage (using Casandra).

Berend de Boer
  • 131
  • 3