0

I have a TPU node name 'demo-tpu' which I am try to delete, using the command gcloud beta compute tpus delete demo-tpu

This results in the following error message:

ERROR: (gcloud.beta.compute.tpus.delete) ABORTED: The resource is already in the process of being deleted

However, when I use the command gcloud beta compute tpus describe demo-tpu, it says the TPU is restarting.

I also tried using the delete option in the GCP console, but it does not seem to work as well. Any suggestions would be helpful.

Boss Caught Me Using S.O.
  • 23
  • 7
Harshit Malik
  • 11
  • 1
  • It would be great if you could provide a [Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve). – NightOwl888 Apr 11 '18 at 10:23

2 Answers2

1

Update: The node did get deleted, but after ~40 minutes, as opposed to the usual 2 minutes. Still no clue as to what caused the problem.

Harshit Malik
  • 11
  • 1
0

The Google Cloud team has since introduced the ctpu tool. It wraps the bringup and deletion ops with ctpu up and ctpu delete. But as you probably have already observed, it takes time to bring up TPU for the first time and delete the VM and/or TPU.

You may also want to double check the "VM instances" and "TPU" tabs in the GCE console to make sure the resources have indeed been deleted.

FYI, you can install ctpu locally which so far works pretty well.

Boss Caught Me Using S.O.
  • 23
  • 7