0

I am using vertex ai's python SDK and it's built on top of Kubeflow pipelines. In it, you supposedly can do this:

train_op = (sklearn_classification_train(
        train_data = data_op.outputs['train_out']
    ).
    set_cpu_limit(training_cpu_limit).
    set_memory_limit(training_memory_limit).
    add_node_selector_constraint(training_node_selector).
    set_gpu_limit(training_gpu_limit)
)

where you can add these functions (set_cpu_limit, set_memory_limit, add_node_selector, and set_gpu_limit) onto your component. I've haven't used this syntax before.

How I can optionally use each 'sub function' only if the variables are specified each function?

For example, if training_gpu_limit isn't set, I don't want to execute set_gpu_limit on the component.

Optimus
  • 1,354
  • 1
  • 21
  • 40

1 Answers1

1

These functions are not appended to the function, but to the component.

In your code, if you do print(type(train_op)) it will be an object type component. So there is no way to add parameters to the function which will influence behavior of the component.

You can do it in pipeline function by changing code to:

train_op = sklearn_classification_train(train_data = data_op.outputs['train_out'])

if training_cpu_limit:
    train_op.set_cpu_limit(training_cpu_limit)

# etc
Optimus
  • 1,354
  • 1
  • 21
  • 40