Stack Overflow
Stack Overflow

Questions tagged [kubeflow]

Kubeflow Training Operator provides Kubernetes custom resources that makes it easy to run distributed or non-distributed TensorFlow/PyTorch/Apache MXNet/XGBoost/MPI jobs on Kubernetes.

GitHub: https://github.com/kubeflow/training-operator

433 questions
2
votes
0 answers

How to deploy Kubeflow Components with cloud build?

I am using Containerized Python components to create the components in my pipeline which works well but I cannot deploy them using Google Cloud Platform Cloud Build. steps: - name: gcr.io/cloud-builders/docker args: [ 'build', '-t', 'dcgp',…
Alexander Townsend
  • 249
  • 2
  • 10
2
votes
1 answer

Mounting an existing volume from a Kubeflow Pipeline: kfp.VolumeOP creates a new volume instead of creating a PVC to existing volume

I am new to KubeFlow and trying to port / adapt an existing solution to run in KubeFlow pipelines. The issue I am solving now is that the existing solution shared data via a mounted volume. I know this is not the best practice for components…
user3197263
  • 71
  • 3
2
votes
2 answers

Return parameters in Kubeflow pipeline component

Im trying to return several parameters from a kubeflow component. Following the documentation it says you should use OutputPath where T is one of the types in python (str, float, int...) so I have this: @component def…
Sergiodiaz53
  • 1,268
  • 2
  • 14
  • 23
2
votes
3 answers

How to test kfp components with pytest

I'm trying to local test a kubeflow component from kfp.v2.ds1 (which works on a pipeline) using pytest, but struggling with the input/output arguments together with fixtures. Here is a code example to illustrate the issue: First, I created a fixture…
Gabriel Caldas
  • 371
  • 2
  • 13
2
votes
1 answer

How to render a confusion matrix visualization in a Kubeflow pipeline component step?

Here is what I am trying from one of the official tutorials import kfp import kfp.dsl as dsl from kfp.components import create_component_from_func @create_component_from_func def confusion_visualization(matrix_uri: str =…
Diablo3093
  • 963
  • 4
  • 15
  • 26
2
votes
3 answers

Failed to pull image

I am following the tutorial here about kserve https://github.com/kserve/modelmesh-serving/blob/main/docs/quickstart.md Is this my docker&k8s issue? I have spent hours trying to debug but to no avail. I am getting the following error: Events: Type…
bonijad383
  • 109
  • 3
  • 7
2
votes
0 answers

Kubeflow Pipelines v2 is giving Permission Denied on OutputPath

In Kubeflow Pipelines v2, running on EKS with default install, I'm getting a "permission denied" error. It ran correctly in KFP v1. time="2022-04-26T21:53:30.710Z" level=info msg="capturing logs" argo=true I0426 21:53:30.745547 18…
aronchick
  • 6,786
  • 9
  • 48
  • 75
2
votes
1 answer

Python OOP in Kubeflow Pipelines

I have a complete preprocessing, training, evaluating ML pipeline in Python 3.8 that I want to transform in a Kubeflow pipeline. It's mainly for segmentation task (i.e. on RGB images) All my code is design in a OOP way. Is this possible to keep the…
mabrtn
  • 21
  • 1
2
votes
1 answer

Kubernetes deployment and replicaset keep reappearing

First time Kubernetes user here. I deployed a service using kubectl -n my_namespace apply -f new_service.yaml It failed, with the pod showing Warning -Back-off restarting failed container I am now trying to delete the failed objects and redeploy a…
Xela
  • 71
  • 8
2
votes
0 answers

storageInitializer InitContainer not starting in Kserve InferenceService

I'm trying out KServe. I've followed installation instructions as per the official docs. I'm trying to create a sample HTTP InferenceService exactly same as this. by doing kubectl apply -f tensorflow.yaml apiVersion:…
chandraSiri
  • 71
  • 6
2
votes
0 answers

Is it possible to get current experiment name in kubeflow pipeline?

When in pipeline code I can use macros like {{workflow.annotations.pipelines.kubeflow.org/run_name}} or {{workflow.name}} to get those values, and potentially use them in some context, eg. passing to some component. What I'd like to do is get…
Marcin Cylke
  • 2,100
  • 2
  • 21
  • 40
2
votes
1 answer

Read the output of a kubeflow pipeline programatically

I run pipelines on Kubeflow with a python command like: client.create_run_from_pipeline_func(pipeline_function, arguments=params_dict[name], run_name=name) It creates a job on Kubeflow pipelines and I would like to be able to access the information…
Robin Nicole
  • 646
  • 4
  • 17
2
votes
0 answers

Timeout problem installing Kubeflow in GCP

After following the documentation to install Kubeflow in Google Cloud Platform, in the step of the kubeflow installation, Kubectl wait for inaccessible resources and waiting long timeouts. In the command make apply kubectl --context=gke-manager wait…
AndreaC
  • 101
  • 7
2
votes
1 answer

Specifying Machine Type in Vertex AI Pipeline

I have a pipeline component defined like this: data_task = run_ssd_data_op( labels_path=input_labels, data_config=config_task.outputs["output_data_config"], training_config=config_task.outputs["output_training_config"], …
James
  • 3,957
  • 4
  • 37
  • 82
2
votes
1 answer

Is it possible to output a list of artifacts of the same type using kubeflow pipelines?

I'm trying to output multiple confusion matrices from a kubeflow pipeline component, like in this example that only outputs one: def eval_model( test_set: Input[Dataset], xgb_model: Input[Model], metrics: Output[ClassificationMetrics], …
VanDough
  • 21
  • 3
1 2 3
…
28 29