0

We have 254 Physical servers when all machines are DELL servers R740.

servers are part of Hadoop cluster. most of them are holding HDFS filesystem and data node & node manager services, part of them are Kafka machines.

The OS that installed on the physical servers is RHEL 7.9

Now we want to add additional 52 physical servers to the cluster. but we can add only the DELL servers R760 That must be installed with RHEL 8.6

each physical server includes 256G and 64 Cores.

The problem is that we can't works with RHEL 8.6 version because Hadoop version. not fit RHEL 8.6

Therefore, we need to stay with RHEL 7.9 version but on other hand DELL server R760 can't be alignment with RHEL 7.9

so, we are thinking to use VM machines instead of DELL R760 physical machines.

it's clear for us that VM performance isn't like the physical machines.

but let's say. if we increase the default memory from 256G to 384G and increase the COREs from 64 to 80 Core's, on VM machine/s

So, in case we set VM machine as above spec with more memory and CPU's

then is it make sense to add the additional VM machines? to Hadoop cluster and not Worry about performance issues?

enter image description here

King David
  • 549
  • 6
  • 20
  • 5
    Running one-one-one legacy RHEL virtually on a more modern RHEL release to achieve hardware compatibility is a fully supported strategy by Red Hat (ask about your specific support/license impact for that though). The overhead / performance penalty for such virtualisation should be relatively small and certainly does not warrant hypervisors with 50% more RAM and 25% more CPU cores than you want to assign to the virtualised RHEL instance. RHEL for example [supports](https://access.redhat.com/articles/rhel-kvm-limits) allocating all physical memory from the host to the virtualised guest. – HBruijn Jul 25 '23 at 16:06
  • you can see from the table that I upload. only RHEL 8.6 can be installed on R760 – King David Jul 25 '23 at 16:11
  • 2
    Hadoop and Kafka server are normally not only required more CPU or RAM, IOPS also a major factor, it's due to kind of job you are executing(multi stage writes and reprocessing). So if you are planning to move to VM make sure to attach faster disks such as nvme disks. – asktyagi Jul 25 '23 at 16:28
  • 2
    But once RHEL 8.6 is installed on that R760 node you can install RHEL's virtualisation software, KVM / libvirt and use that launch a guest, a VM, that runs your legacy RHEL 7. With one-on-one virtualisation you will get almost all performance from the physical server in your guest, you can tune and optimise and directly assign devices to your guest, without any abstraction layers and you'll get close to similar performance as running rhel 7 baremetal – HBruijn Jul 25 '23 at 16:38
  • appreciate, if you can post an answer about my Question. , I think it will be better to explain. your recommendation – King David Jul 25 '23 at 17:50

1 Answers1

3

Required note: I work for Dell

@HBrujin's comment already has basically answered the question and is sufficiently thorough that it is worth repeating:

Running one-one-one legacy RHEL virtually on a more modern RHEL release to achieve hardware compatibility is a fully supported strategy by Red Hat (ask about your specific support/license impact for that though). The overhead / performance penalty for such virtualisation should be relatively small and certainly does not warrant hypervisors with 50% more RAM and 25% more CPU cores than you want to assign to the virtualised RHEL instance. RHEL for example supports allocating all physical memory from the host to the virtualised guest.

^that's what I would do.

What I will add is that I frequently work with HPC customers and at some point you need to bite the bullet. RHEL 7 is a decade old now and is basically EOL (it technically has some bare minimum maintenance until next June at which point you're completely on your own). Working in HPC I fully appreciate the effort that it takes to upgrade, but if you're at the point that you're buying new hardware backwards compatibilty barring running everything in VMs is only going to become increasingly cumbersome. NVMe doesn't behave quite right in some cases, BIOS problems have arisen, driver support is increasingly dodgy, I've seen all sorts of things start breaking on our Gen 15 servers let alone Gen 16 (what you have).

If one isn't already in place, I strongly recommend creating a migration plan to a newer version of RHEL.

Grant Curell
  • 1,043
  • 6
  • 19