3

I am trying to follow an official tutorial on training model with google cloud TPUs. This is the tutorial : https://cloud.google.com/tpu/docs/tutorials/transformer-pytorch

At the step "Launch a Cloud TPU resource" i do the following

:~$ gcloud compute tpus create train-bert-one \
> --zone=europe-west4-a \
> --network=default \
> --version=pytorch-1.6 \
> --accelerator-type=v3-8

Just as in the tutorial, I merely adapted the region.

The command fails with

ERROR: (gcloud.compute.tpus.create) PERMISSION_DENIED: Permission 'tpu.nodes.create' denied on 'projects/bert-archaea-fine-tuning/locations/europe-west4-a/nodes/train-bert-one'

Does anyone know this error? Cannot find anything on it.

fteufel
  • 59
  • 1
  • 5

1 Answers1

1

Google Support was able to fix it, the problem was that for some reason there was no working API service account created automatically when creating the project with

gcloud compute --project=${PROJECT_ID} instances create transformer-tutorial \
--zone=us-central1-a  \
--machine-type=n1-standard-16  \
--image-family=torch-xla \
--image-project=ml-images  \
--boot-disk-size=200GB \
--scopes=https://www.googleapis.com/auth/cloud-platform

Was fixed with adding the flag --service-account= to the command and manually specifying which service account to use.

fteufel
  • 59
  • 1
  • 5