4

scenario: master: x1 machine type workers: x2-machine type, x3-machine type.

For the above scenario: AWS EMR instance fleet allows users to create different worker instance types. From dataproc console, I noticed options is for only, N-worker types(all worker types of same type). Does Dataproc HDI also support this option of creating different instance types for worker?

user2622678
  • 393
  • 1
  • 7
  • 17

1 Answers1

1

Dataproc supports 2 types of worker group: (primary) workers and secondary workers. But they must have the same machine type as of Q2 2022, there is only one flag --worker-machine-type for it.

Note that it does allow secondary workers to have different disk type, disk size, accelerator, etc. See this doc for more details.

Dagang
  • 24,586
  • 26
  • 88
  • 133