5

Our new DAS unit is on the horizon and I'm looking for the best way to configure the storage.

We currently use vSphere 6.0 but will have to upgrade to 6.5 when Update 1 is released as all HDDs in the new DAS are 512e (sector size is 4096 emulated as 512) which is only supported in 6.5.

The total capacity of two Dell MD1280 (populated with 10TB HDDs) will greatly exceed 64 TB max size of Datastore, many times. Depends on the number of hard drives we decide to use/buy.

Initially I thought about creating multiple Virtual Disks in iDRAC

RAID5 -> 7 x 10TB = 60TB with 1 drive failure

Then create Datastore and Virtual Disks with vCenter and then span them with LVM on the VM (all VMs are Linux)

This is the same and only option I was given by DELL VMWare tech support.

The problem here is that if I fully populate the both units, I would lose 24 drives (240TB) for parity and still only one of them can fail per each Virtual Disk. Any other RAID level only adds to the loses.

I used RDM approach (mounting physical(VDs from iDRAC) drives on VMs) in the past while running vSphere on a Free licence to work around max 4TB Virtual Disk size. I wonder if that would work and what would be the possible downsides of this approach, if it works on this scale.

I know I might be forced to do this outside of VMWare and be able to just span all HDDS into a huge array with multiple parity disks but wanted to ask if anybody has some word of wisdom in this subject. Details of this approach should probably be discussed in a separate question, if there isn't one already.

Any advice would be greatly appreciated.

Bart C
  • 211
  • 1
  • 2
  • 8
  • Hmm, what exactly are you trying to do here, do you need a single 64TB vmdk or do you just want a huge datastore? – Stuggi Nov 19 '16 at 11:07
  • In short I need a solution allowing me to hook-up up to 1.6 PB to a VM in a way where I can lose up to 6 HDDs without losing data across the whole 1.6 PB array. – Bart C Nov 21 '16 at 07:45
  • And only one VM needs this 1.6PB array? I'd connect it through iSCSI directly to the VM, as we do with backup drives – Stuggi Nov 21 '16 at 10:29
  • Thanks Stuggi, could you elaborate on this. I though you are talking about passthrough RDM approach but according to [this](https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2003813) the limit still stands at 64TB. – Bart C Nov 21 '16 at 11:57
  • 1
    Will you please answer Stuggi's question - there's really only one important question here - do you need any single VM to have access to more than 62TB - also PLEASE do not use RAID5 for this, you WILL lose data any time you need to replace a disk. – Chopper3 Nov 23 '16 at 19:00
  • "WILL" is a strong word but the data loss risk is very real with RAID5 with big disks. – Dan Pritts Nov 23 '16 at 19:13
  • Chopper3, I need only one VM to access this storage and I need much more than 62TB. As I explained in my question, I would lose at least 24 drives for parity by grouping them in 60TB VDs. Not an option. I need one huge array with multiple fails until I lose data. RAID5 is not an option, I only mentioned that to explain what my problem is. – Bart C Nov 23 '16 at 21:21

2 Answers2

6

Yes. PCI Passthrough is what you need. RDM won't help you since it has a limitation of 64TB per device.

Check this article for more info: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1010789

Strepsils
  • 5,000
  • 10
  • 14
0

Map up the large drives using iSCSI inside the VM. Basically you create the VM in VMware with just an OS drive. Then inside the VM you use the appropriate software to map the iSCSI targets. In Windows that would be iSCSI Initiators. That way VMware never has to deal with the large array and you don't have to care about the maximums in VMware. You still however need to pick an OS and filesystem that supports the size of volume you are after. You can also just use this on one VM, if you don't use a cluster-aware file system.

Stuggi
  • 3,506
  • 4
  • 19
  • 36
  • 1
    Thanks Stuggi, I will look into this. Sounds promising. I guess after creating a Virtual Disk as a RAID array in iDRAC I will need to configure hardware passthrough on ESXi host to pass on this device straight to the VM. I suppose I can test this part with a smaller iDRAC Virtual Drive. – Bart C Nov 22 '16 at 15:57
  • 1
    That's also maybe an option, but not what I suggested, I didn't notice that this array was DAS and not some sort of SAN. – Stuggi Nov 23 '16 at 06:52
  • 1
    It's a shame that this storage wasn't bought to form a NAS that was then presented to the ESXi host/s as NFS, that'd have been a far simpler - though more expensive - option. – Chopper3 Nov 23 '16 at 19:33