Stack Overflow
Stack Overflow

Questions tagged [tfx]

TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines

TFX is a Google-production-scale machine learning platform based on TensorFlow. It provides a configuration framework and shared libraries to integrate common components needed to define, launch, and monitor your machine learning system.

Resources

185 questions
3
votes
2 answers

Kubeflow Pipeline RuntimeError while running multiple trainers with tfx

I like to have multiple trainers running simultaneously using the same ExampleGen, Schema and Transform. Below is my code adding extra components as trainer2 evaluator2 and pusher2. But I've been getting the following error, and I'm not sure how to…
LLTeng
  • 385
  • 3
  • 4
  • 15
3
votes
1 answer

To load pb file : DecodeError: Error parsing message

I'm trying to load a .pb file generated from pusher tfx pipeline. I'm using the following function to load the file, but I got the following error from the function. Please help. error: in load_model() 2 …
LLTeng
  • 385
  • 3
  • 4
  • 15
3
votes
2 answers

TFX StatisticsGen for image data

Hi I've trying to get a TFX Pipeline going just as an exercise really. I'm using ImportExampleGen to load TFRecords from disk. Each Example in the TFRecord contains a jpg in the form of a byte string, height, width, depth, steering and throttle…
Joshua Patterson
  • 439
  • 5
  • 12
3
votes
1 answer

TensorFlow Serving export signature without arguments

I would like to add extra signature to SavadModel, which will return business description and serve it with TensorFlow Serving. @tf.function def info(): return json.dumps({ 'name': 'My model', 'description': 'This is model…
Andras Gyacsok
  • 61
  • 7
3
votes
1 answer

Which TFX orchestrator is de facto tandard for TFX?

I am beginner with tensorflow and now in a project where I need to deploy distributed production platform for tensorflow. I appreciate if I could get some help to clarify my thought. Reading the online doument, and youtube, I understood that main…
Yu Watanabe
  • 621
  • 4
  • 17
3
votes
1 answer

Using TFX for designing image piplines

When reading the documentation for TFX, especially in the parts related to pre-processing of the data, I would think the pipeline design is more appropiate for categorical features. I wanted to know whether TFX could also be used for pipelines…
Siavash S
  • 43
  • 4
2
votes
4 answers

Why we need TFX if we have airflow for orchestration

I still don't get why we need TFX. TFX will convert your defined pipeline to Airflow DAG and run it on airflow, I could just write my pipelines in python and use Airflow's PythonOperator to build a pipeline directly right? why bother learning…
ariverhorse
  • 31
  • 2
2
votes
2 answers

Image processing in Tensor flow TFX pipelines

I am trying to get a Tensorflow TFX pipeline up and running using the MNIST dataset. # Imports import pandas as pd import numpy as np from keras.datasets import mnist import tensorflow as tf from tfx import v1 as tfx import os from tfx.components…
tim_ai
  • 41
  • 5
2
votes
2 answers

Using String parameter for nvidia triton

I'm trying to deploy a simple model on the Triton Inference Server. It is loaded well but I'm having trouble formatting the input to do a proper inference request. My model has a config.pbtxt set up like this max_batch_size: 1 input: [ { …
Regalia
  • 129
  • 2
  • 10
2
votes
1 answer

Do TFX pipelines support training on TPUs?

I am new to TFX and I would like to know if it's possible to create a TFX pipeline that can train Keras models on TPUs nodes using TPUStrategy from a TPUClusterResolver. Looking at TFX documentation, it is not clear to me if such is possible. Any…
TheDude
  • 51
  • 1
  • 4
2
votes
0 answers

TensorFlow Extended installing error with some conficts

I need help installing tfx, I just ran this command: pip install -U tfx But I get this error: ERROR: Cannot install tfx==0.13.0, tfx==0.14.0, tfx==0.15.0, tfx==0.21.0, tfx==0.21.1, tfx==0.21.2, tfx==0.21.3, tfx==0.21.4, tfx==0.21.5, tfx==0.22.0,…
Daniel Díaz
  • 186
  • 1
  • 2
  • 12
2
votes
1 answer

TFX runs out of memory on 32GB of RAM with only a 232.9MB TFRecord file. Is there something wrong with my code?

I am trying to train a model that detects whether someone is using sunglasses using tfx and a subset of the celebsA dataset (~ 26k images). I have written the images and labels to a TFrecord that is 232.9MB. When I then go through the different…
Dariyan Khan
  • 79
  • 1
  • 6
2
votes
2 answers

Specify machine type for a single TFX pipeline component in Vertex AI

I'm using TFX to build an AI Pipeline on Vertex AI. I've followed this tutorial to get started, then I adapted the pipeline to my own data which has over 100M rows of time series data. A couple of my components get killed midway because of memory…
Simon Corcos
  • 962
  • 14
  • 31
2
votes
1 answer

Loop over a tensor in Tensorflow

I want to loop over a Tensorflow tensor, my code is something like this: elements = tf.constant([1,2,3]) x = tf.constant([1.000001, 1.1, 2.1, 2.00004, 3.001]) EPSILON = 0.0001 for elem in elements: mask = tf.experimental.numpy.isclose(x, elem,…
eng2019
  • 953
  • 10
  • 26
2
votes
0 answers

TensorFlow Extended data_accessor.tf_dataset_factory() shape discrepancies

I am facing a perplexing issue while attempting to convert a vanilla tensorflow/keras workflow into a tensorflow extended pipeline. In short: the datasets generated using tfx’s ExampleGen component have different shapes from those created manually…
ohno
  • 31
  • 2
1
2
3
…
12 13