2

I am using https://dataproc.googleapis.com/v1/projects/{projectId}/regions/{region}/clusters to create GCP Dataproc clusters as described at https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.clusters/create.

I am using service account credentials that have been exported into a JSON keyfile. That service account (myserviceaccount@projectA.iam.gserviceaccount.com) exists in projectA and I have been able to use it to successfully create Dataproc clusters in projectA.

I now need to use the same service account to create Dataproc clusters in projectB. I'm running exactly the same code using exactly the same credentials, the only difference is the project that I'm creating it in. I have granted myserviceaccount@projectA.iam.gserviceaccount.com the exact same permissions in projectB as it has in projectA but when I try and create the cluster it fails:

2019-03-22 10:58:47 INFO: _retrieve_discovery_doc():272: URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dataproc/v1/rest
2019-03-22 10:58:54 INFO: method():873: URL being requested: GET https://dataproc.googleapis.com/v1/projects/dh-coop-no-test-35889/regions/europe-west1/clusters?alt=json
2019-03-22 10:58:54 INFO: new_request():157: Attempting refresh to obtain initial access_token
2019-03-22 10:58:54 DEBUG: make_signed_jwt():100: [b'blahblahblah', b'blahblahblah']
2019-03-22 10:58:54 INFO: _do_refresh_request():777: Refreshing access_token
2019-03-22 10:58:55 WARNING: _should_retry_response():121: Encountered 403 Forbidden with reason "forbidden"

So, that service account is forbidden from creating clusters in projectB, but I don't get any information about why. I am hoping there are some audit logs that explain more about why the request was forbidden but I've looked in https://console.cloud.google.com/logs/viewer?project=projectB and can't find any.

Can someone tell me where I can get more information to diagnose why this request is failing?

Igor Dvorzhak
  • 4,360
  • 3
  • 17
  • 31
jamiet
  • 10,501
  • 14
  • 80
  • 159
  • 2
    One thing you could do is setup gcloud to use the service account. Running gcloud commands with --log-http may give additional information. – tix Mar 22 '19 at 17:52
  • Thx, I’ll try that – jamiet Mar 22 '19 at 18:00
  • I just tried this and was able to create clusters in my projectA and projectB using a service account created in projectA and added to projectB with roles/dataproc.admin. Have you associated billing and enabled the dataproc service in projectB? – DazWilkin Mar 23 '19 at 04:30

1 Answers1

4

As mentioned in the comments, one way to get more information on the failed request is to set up gcloud to use the service account. Running gcloud commands with --log-http may also give additional information.

Re-pasting here for easier readability/visibility.

Malina
  • 98
  • 4