7

When I trying to create the cluster with 1 master and 2 data nodes, I am getting below error:

Cannot start master: Insufficient number of DataNodes reporting

Worker test-sparkjob-w-0 unable to register with master test-sparkjob-m. This could be because it is offline, or network is misconfigured.

Worker test-sparkjob-w-1 unable to register with master test-sparkjob-m. This could be because it is offline, or network is misconfigured.
James Z
  • 12,209
  • 10
  • 24
  • 44
Skumar
  • 71
  • 2
  • 3
    Check the network configuration (firewall rules) as you must ensure that internal communication is possible. You can find the docs [here](https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network). If needed, remove the conflicting rules or create a new VPC network and associate it to the cluster upon creation. – Guillem Xercavins Jan 21 '18 at 19:28
  • 1
    Did you add any tags when you create your cluster? [Tags are used to attach firewall rules to each node.](https://codelabs.developers.google.com/codelabs/cloud-dataproc-gcloud/index.html#9) Cluster name needs to prefix match your network tags, otherwise the spark VMs will not have correct firewall rules. – Max Yuan Apr 20 '18 at 15:38
  • + 2cents on above replies: check if your initialization scripts (in the case you have them) are not stuck. – Alan Borsato Mar 03 '20 at 19:33

0 Answers0