Stack Overflow
Stack Overflow

Questions tagged [kubeflow-pipelines]

Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows based on Docker containers.

273 questions
0
votes
2 answers

Can a particular component(step) in a kubeflow pipeline be scaled(autosclaed) horizontally?

All this while I was using lightweight python function as a component in kubeflow pipeline but recently I switched to use docker container as a component in kubeflow pipeline which goes something like this... produce: str = 'hello' ): return…
Sagar SN
  • 35
  • 1
  • 8
0
votes
1 answer

Kubeflow pipeline error "module 'kfp.dsl' has no attribute 'RUN_ID_PLACEHOLDER'"

I am running this Kubeflow pipeline example below in Jupyter NB. In the def gh_summ() part, it gave me an error message: "module 'kfp.dsl' has no attribute 'RUN_ID_PLACEHOLDER'". Any suggestions? Thank you! The Kubeflow pipeline Jupyter nb: Need to…
v-loves-avocados
  • 11
  • 1
  • 4
0
votes
1 answer

Install Custom Dependency for KFP Op

I'm trying to setup a simple KubeFlow pipeline, and I'm having trouble packaging up dependencies in a way that works for KubeFlow. The code simply downloads a config file and parses it, then passes back the parsed configuration. However, in order to…
James
  • 3,957
  • 4
  • 37
  • 82
0
votes
0 answers

How can I upgrade my existing AI platform-pipelines deployment to a newer version?

I currently have a deployment of AI platform-pipelines v0.2.5 running. I saw that 8 days ago, a new version v0.5.1 got added to the container registry. There have been a lot of changes, fixes, etc.. between these versions and I would like to update…
Jonas Boecquaert
  • 51
  • 6
0
votes
1 answer

TFX running multiple trainers simultaneously

I'm new to tfx and learning to put together the pipeline. I've successfully built a pipeline in Kubeflow on GCP. I like to know how can I run multiple trainers that will generate different outputs/pushers in the pipeline using the same…
LLTeng
  • 385
  • 3
  • 4
  • 15
0
votes
1 answer

Kubeflow fails to deploy using both CLI and Console

I deleted my KF cluster last night to create a new one (using kubectl cluster command not Kfctl delete), and then when I tied to create a new one, it fails, it does not work with CLI not Console. I found other people have run into this issue before,…
Hamid Hassanzadeh
  • 1
0
votes
1 answer

Destructor for kfp pipelines

I'm working on kubeflow(kf) pipelines that calls other kf pipelines. We use a top level parent pipeline, to call out multiple child pipelines in parallel. During development and testing it is often the case where we would like to kill a complete…
GeneralBecos
  • 2,476
  • 2
  • 22
  • 32
0
votes
1 answer

Can Kubeflow Pipelines run GPU components in parallel?

I am trying to build a kubeflow pipeline where I run two components (with a GPU constraint) in parallel. It seemed like a non-issue, but every time I tried it, one component would get stuck at "pending" until the other component is done. Example…
RawChicken
  • 31
  • 7
0
votes
2 answers

Where is dockershim and related logs for debugging kubeflow/pipeline/helloworld

I try to run kubeflow samples (pipelines/pytorchjob etc) but it stacks as ContainerCreating. For this purpose, I want to see dockershim and docker logs. Is there any documentation? sample code is…
Atsushi Sakai
  • 189
  • 2
  • 7
0
votes
1 answer

Handle credentials in a kubeflow's ContainerOp

I'm trying to run a kubeflow pipeline setup and I have several environements (dev, staging, prod). In my pipeline I'm using kfp.components.func_to_container_op to get a pipeline task instance (ContainerOp), and then execute it with the appropriate…
torpido
  • 63
  • 9
0
votes
1 answer

Running XGBoost example in MiniKF (Google Cloud Platform)

I'm trying to run XGBoost example pipline in MiniKF installed in GCP through application manager. I have a free account with free credits and apparently the max number of CPUs for this account is 8. MiniKF was installed and other example pipelines…
Konstantin
  • 2,937
  • 10
  • 41
  • 58
0
votes
1 answer

How can I make an API after compiling and running my pipeline on Kubeflow?

I built a pipeline which takes an image and returns a number of persons. I want to make an API which takes an image and returns a JSON file with count using Kubeflow.
Omar_Medhat
  • 11
  • 1
  • 2
0
votes
1 answer

Kubeflow pipelines storage access error inside same GCP Project?

After installing Kubeflow 0.7 on a new GKE Cluster (via https://deploy.kubeflow.cloud) I have configured OAuth and Workload Identity based on their respective tutorials. On my pipeline I have to access a GCS bucket from the same project and it seems…
Claudio Davi
  • 154
  • 1
  • 9
0
votes
2 answers

How to upgrade an existing kubeflow pipeline?

In the pipeline UI I use the upload pipeline button to upload new pipelines. Since the pipeline name is unique, the only way to update is to delete the old pipeline and then to upload a new one. Is there a better way, maybe to manage a version…
user3599803
  • 6,435
  • 17
  • 69
  • 130
0
votes
0 answers

How to attach GCP secret to Kubernetes service account?

How can i use a secret object created from Google cloud JSON file into a service account? I have minikf on the VM and kubeflow installed. I am trying to make a container using Jupyter notebook in minikf Kubernetes cluster. The notebook has access to…
Pankaj Kumar
  • 3,139
  • 1
  • 15
  • 9
1 2 3
…
18
19