Stack Overflow
Stack Overflow

Questions tagged [kubeflow-pipelines]

Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows based on Docker containers.

273 questions
0
votes
1 answer

kubeflow stuck in pipeline starting: Unable to attach or mount volumes: unmounted volumes=[mlpipeline-minio-artifact]

kubernetes and kubeflow were all set on my GPU service, while the errors occurred when I tried to start up a tutorials pipeline, it looks like stuck in something about pvc and pv. here are the details: ... Environment Variables from: …
秋水泡茶
  • 7
  • 1
0
votes
1 answer

Vertex Pipeline: CustomPythonPackageTrainingJobRunOp not supplying WorkerPoolSpecs

I am trying to run a custom package training pipeline using Kubeflow pipelines on Vertex AI. I have the training code packaged in Google Cloud Storage and my pipeline is: import kfp from kfp.v2 import compiler from kfp.v2.dsl import component from…
Christopher Larsen
  • 11
  • 4
0
votes
1 answer

Using ImagePullSecrets with Kubernetes

I'm trying to support two systems both of which pull images from private repos. One is from Kubernetes - which needs imagePullSecrets via a mounted secret and the other just needs standard docker login. Based on this - Pulling images from private…
aronchick
  • 6,786
  • 9
  • 48
  • 75
0
votes
1 answer

Not being able to inference TensorFlow model hosted by kfserving component in Kubeflow

Hi so I am using kfserving v.0.5.1 component for hosting the model. I am able to download and deploy model from s3 but facing issue when try to access it. After deployment kfserving outputted the following…
Ahsan Naseem
  • 1,046
  • 1
  • 19
  • 38
0
votes
1 answer

TFX Evaluator does not run in Dataflow so it fails due to lack of memory for the pod

I am running a pipeline in AI Platform pipelines based on TFX. All components run fine until the Evaluator. It just does not want to run on Dataflow, it runs in the Kubeflow pod, so it fails as there is no enough memory in there. Apache Beam config…
Gorjan Todorovski
  • 61
  • 4
0
votes
1 answer

Kubeflow on Azure - Can't run a pipeline

I installed Kubeflow on Azure following this Guide However, when I create an experiment and then try to run Taxi Pipeline, the following error appears: MountVolume.SetUp failed for volume "docker-sock" : hostPath type check failed:…
pablofiumara
  • 1,747
  • 1
  • 16
  • 24
0
votes
2 answers

What is the correct way for share dataframes between components?

I am working with a legacy project of Kubeflow, the pipelines have a few components in order to apply some kind of filters to data frame. In order to do this, each component downloads the data frame from S3 applies the filter and uploads it into S3…
Tlaloc-ES
  • 4,825
  • 7
  • 38
  • 84
0
votes
1 answer

How to specify task pod resources in Kubeflow Kale

Is it possible to specify the pod resources (memory and CPU requests and limits), for a task in a kubeflow pipeline created with Kale? To provide more details, when writing the kubflow pipeline in python using the DSL, I can specify the task pod…
Czeslaw
  • 116
  • 1
  • 7
0
votes
0 answers

GKE upload to GCS fails (Error 403: insufficient permission), if more than one Pod is running on the same GKE node (VM)

First of all, I want to emphasize that this is not a duplicate of (e.g.) this one. Problem description: I am running Kubeflow Pipelines (set up on GCP AI Platform Pipelines) on a GKE Cluster. Each pipleine consists of several components (i.e. docker…
Peabuddy
  • 1
  • 1
0
votes
1 answer

KFServing pod "error: container storage-initializer is not valid"

I am new to KFServing and Kubeflow. I was following https://github.com/kubeflow/kfserving/tree/master/docs/samples/v1alpha2/tensorflow to deploy a simple inference service. However, when looking at the logs, I am unable to find the container…
franco mo
  • 11
  • 1
0
votes
1 answer

Connect to a gRPC service on a GKE cluster from Cloud Build

We’re using a managed Kubeflow Pipelines (KFP) instance, created with GCP’s AI Platform Pipelines, with a managed MySQL instance, created with Cloud SQL. We’re also using Cloud Build to build and run our KFP pipelines. We’d like to add a Cloud Build…
hahns0lo
  • 1
  • 2
0
votes
0 answers

How can I concatenate Tensorflow Dataset columns?

I have a Keras model that takes an input layer with shape (n, 288, 1), of which 288 is the number of features. I am using a TensorFlow dataset tf.data.experimental.make_batched_features_dataset and my input layer will be (n, 1, 1) which means it…
Hosein
  • 11
  • 2
0
votes
1 answer

How do I format the response body of a request to the predict endpoint of a Google Cloud Platform API request

I'm working off of the tutorial that is premade on GCP when working with the AI Platform component. In short it involves using Kubeflow and Tensorflow Extended to build and deploy a completed ML model for predicting who will be a big tipper off of…
Damisco
  • 85
  • 7
0
votes
2 answers

Issue when trying to pass data between Kubeflow components using files

I made two components using python functions and I am trying to pass data between them using files, but I am unable to do so. I want to calculate the sum and then send the answer to the other component using a file. Below is the partial code (The…
Franco
  • 441
  • 3
  • 18
0
votes
1 answer

How to mount a Google Bucket in Kubeflow Pipeline?

I have a KubeFlow Pipeline up and running on a VM in GCP with KF. I create the pipeline using a Jupyter Notebook server with image jupyter-kale and python. The first part of the pipeline is doing the dataprep, it downloads images and saves them to a…
rambutan
  • 199
  • 2
  • 10
1 2 3
…
18 19