1

I have a Data Fusion Pipeline that reads a GCS Bucket, do some transformation and then store the output (Sink) in another Cloud Storage Bucket. However, I am getting below exception when the Pipeline runs. I have granted Cloud Storage Admin role to Data Fusion Service account as well as the Dataproc service account.

2021-10-18 17:28:09,629 - WARN  [provisioning-task-1:i.c.c.r.s.c.DataprocUtils@88] - GCS path cdap-job/4f61e40e-3038-11ec-b538-e22acad5362e was not cleaned up for bucket gs://df-3070032220784195332-e6pu33jqaii6zerkaizbbqaaaa due to The specified bucket does not exist.. 
com.google.cloud.storage.StorageException: The specified bucket does not exist.

Any pointer would be highly appreciated. Thank you!

Ricco D
  • 6,873
  • 1
  • 8
  • 18
Santosh Kumar Sahoo
  • 47
  • 6

1 Answers1

1

We can fix it by manually recreating the GCS bucket. I also made the same mistake and recreation of the bucket helped me to resolve it.

SKumar
  • 43
  • 6