0

I am trying to create Dataproc cluster on GKE. I was following steps on the GCP official website.

After running the following command on gcloud CLI,

DP_CLUSTER="test-gke" \
  REGION="asia-east2-a" \
  GKE_CLUSTER="airflow-cluster" \
  GKE_CLUSTER_NAMESPACE="spark" \
  BUCKET="test-spark" \
  DP_POOLNAME="dataproc-pool"

gcloud dataproc clusters gke create $DP_CLUSTER \
    --region=$REGION \
    --gke-cluster=$GKE_CLUSTER \
    --spark-engine-version=latest \
    --staging-bucket=$BUCKET \
    --pools="name=$POOL_NAME,roles=default" \
    --setup-workload-identity

I have encountered the error:

ERROR: (gcloud.dataproc.clusters.gke.create) INVALID_ARGUMENT: Multiple validation errors:
 - GKE location 'asia-east2-a' is not supported by this Dataproc region.
 - Global region is not supported for Dataproc Virtual Cluster.

On its official website, it says Dataproc is available across all regions and zones of the GCP.enter image description here

Is there any way to solve this error?
Any comments would be appreciated.

yw_9
  • 29
  • 3

1 Answers1

1

Cloud Dataproc does support all regions and zones. However, in your case while specifying the environment variable region you are assigning it to a zone ie. asia-east2-a. Specifying the REGION as asia-east2 should solve your error.

Sakshi Gatyan
  • 1,903
  • 7
  • 13