0

we are looking for some help deciding what hardware to buy to support an internal Hadoop cluster. My company currently uses 1 dedicated server for Hadoop which has 196GB ram and 24cores and 6 1TB SATA hard drives. We are wanting to scale up our capacity but are unsure what to buy for the best value. We are looking to spend around £10k on equipment and have considered blades such as dell PE m710 where we can get 8 blades each with 12 cores, 288 GB ram and 4x600 GB SAS drives with enclosure for our budget. Is this a good use of the money available or would better spec newer servers be more appropriate?

Many thanks for any help

TWith2Sugars
  • 113
  • 1
  • 5

1 Answers1

1

dell PE m710 where we can get 8 blades each with 12 cores, 288 GB ram and 4x600 GB SAS drives

  • Hadoop node should be cheap(supermicro for example) and have 2U size.
  • 600 GB SAS - bad choice. You need something like 12x4GB SATA 7200rpm w/o RAID
  • The most important thing is a network. You should use 10Ge or 1G with Bonding.

Please read this: https://blog.cloudera.com/blog/2013/08/how-to-select-the-right-hardware-for-your-new-hadoop-cluster/

Alexander Makarenko
  • 693
  • 4
  • 10
  • thanks for the reply and I agree that more disks would be better however we are using this mainly for spark which is memory + cpu once you've read in the data. As per the cloudera doc the 'compute intensive' machine rec we are thinking more memory and fewer disks. – TWith2Sugars Aug 16 '17 at 07:02
  • Does anyone have a similar use case where they would be happy to share their set up? – TWith2Sugars Aug 16 '17 at 07:02