0

For the past two years, I've been testing the performance of our software in the Google Compute Engine (GCE), using up to a total of 1k vCPUs for the worker-VMs and the VMs for MySQL NDB Cluster that we use.

I have automated the creation of worker VMs (that run our software) with the help of templates and instance groups, but for the MySQL NDB cluster I've always had two fixed machines which I occasionally resized manually (i.e. change the vCPUs and RAM).

I am now trying to bring up a variable number of VMs for the NDB cluster with different amounts of vCPUs and RAM, and I'd like to automate this process in my test suite. The main problem is that the NDB config expects fixed HostNames in the config. To use GCE templates and instance groups, I would be able to bring up only dynamically named instances like ndb-1ffrd, ndb-i8fhh, ....

To exemplify, here is my current ndb config (one of many) that involves fixed VMs ndb1 and ndb2:

[ndbd]
HostName=ndb1
NodeId=1
NodeGroup=0
[ndbd]
HostName=ndb2
NodeId=2
NodeGroup=0
[ndbd]
HostName=ndb1
NodeId=3
NodeGroup=1
[ndbd]
HostName=ndb2
NodeId=4
NodeGroup=1

I'd like to convert the fixed VMs ndb1/ndb2 into GCE instance groups where I can choose to bring an arbitrary number of such instances up (typically 2 or 4 though) for a test and then destroy the VMs afterwards, and perform this on-demand in an automated fashion during my tests. Reasoning behind this is to have repeatable tests with differently configured VMs. Changing many parameters manually over several tests leads to a nightmare when figuring out what the exact configuration was 10 tests ago -- this way each test would refer to a specific instance template for the ndb VMs.

However, GCE instance group members have a random suffix in their name and the NDB config expects fixed HostNames or IPs. So I'd need to either:

  • Have GCE generate instances from instance groups named in a deterministic way (e.g. ndb1, ndb2, ndb3, ...), so that I can rely on those names in my ndb configs, or
  • Somehow allow arbitrary hosts (or hosts with arbitrary suffixes) to connect as ndb nodes but still make sure that the same host isn't added to the same NodeGroup more than once -- something that is manually ensured in the above sample config.

Is there any way to achieve what I'm trying to do?

Irfy
  • 9,323
  • 1
  • 45
  • 67

1 Answers1

-1

I think that this may be achieved using scripts along with gcloud SDK, gcloud let's you to launch resources in GCP, there are also many options/flags that may be helpful to set the configurations that you need.

Hope this helps.

Adad O.
  • 97
  • 5
  • Hello Adad and welcome to StackOverflow. Unfortunately, your answer is not helpful at all. Consider stating more precisely, how my goal could be achieved. For example, state which options allow me to dynamically resize instance groups with a fixed/deterministic naming scheme. `gcloud compute instance-groups managed create` only supports a `--base-instance-name` option with no possibility of customizing the random string generated for the actual instance names. – Irfy Feb 11 '19 at 22:18