0

I tried running thousands of machines using the genomics pipeline using the preemptible flag in v2alpha1 JSON mapping.

Even though the machines were preemptible - a lot of workers were using up persistent disk space while not even having started.

the gcloud alpha genomics operations describe $operation_id

I see description: Worker released

    details:
      '@type': type.googleapis.com/google.genomics.v2alpha1.WorkerReleasedEvent
      instance: google-pipelines-worker-10f2002aa213b3108fb69a7488d0d4ce
      zone: us-east1-c
    timestamp: '2019-04-15T01:49:29.065576Z'
  - description: Worker "google-pipelines-worker-10f2002aa213b3108fb69a7488d0d4ce"
      assigned in "us-east1-c"
    details:
      '@type': type.googleapis.com/google.genomics.v2alpha1.WorkerAssignedEvent
      instance: google-pipelines-worker-10f2002aa213b3108fb69a7488d0d4ce
      zone: us-east1-c
    timestamp: '2019-04-14T19:16:08.141993Z'

I expected workers to be assigned only when a preemptible instance was available. It looks like the assigned workers took up disk space without taking up cpu resources.

Is there something more I should be doing - when setting up the pipeline json.

https://cloud.google.com/genomics/reference/rest/Shared.Types/Metadata#Pipeline

Bharathwaj Raghu
  • 1

1 Answers1

0

Could you please join and post this question to the Google Genomics Discuss mailing list and provide some more detail with what you're seeing here? Would like to understand what pipeline you're trying to run, the config you're using, and where you're seeing the workers taking up storage without CPU.

a m
  • 11
  • 2