Stack Overflow
Stack Overflow

Questions tagged [tfx]

TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines

TFX is a Google-production-scale machine learning platform based on TensorFlow. It provides a configuration framework and shared libraries to integrate common components needed to define, launch, and monitor your machine learning system.

Resources

185 questions
2
votes
1 answer

How to connect 2 pods in kubernetes as they were in the same local-net with all the ports opened

TLDR; It's possible to connect 2 pods in Kubernetes as they were in the same local-net with all the ports opened? Motivation Currently, we have airflow implemented in a Kubernetes cluster, and aiming to use TensorFlow Extended we need to use Apache…
Giovani Merlin
  • 125
  • 8
2
votes
2 answers

Remove stopwords in Tensorflow extended

I have to preprocess NLP data, so I've to remove the stopwords (from nltk library) from a Tensorflow dataset. I tried many thing like this: docs = tf.data.Dataset.from_tensor_slices([['Never tell me the odds.'], ["It's a trap!"]]) tokenizer =…
tCot
  • 307
  • 2
  • 7
2
votes
0 answers

How would one go about creating a custom TFX component to perform feature engineering/building

TFX Transform component operates with Tensors that cannot be executed eagerly within its preprocessing_fn. Therefore the values for each feature cannot be passed to a python function that generates new features based on those values. I tried…
MachineLearner
  • 21
  • 2
2
votes
2 answers

TFX / Tensorflow Data Validation (TFDV) does not work on data drift

I'm not sure what the problem is with the code below. I read the documentations and all of them point to an approach similar to this one. Here is a simple example that doesn't work. My expectation was a notice for the feature x1 since the…
John Tartu
  • 41
  • 2
2
votes
1 answer

TensorFlow Extended | Trainer Not Warm Starting With GenericExecutor & Keras Model

I'm presently trying to get a Trainer component of a TFX pipeline to warm-start from a previous run of the same pipeline. The use case is: Run the pipeline once, produce a model. As new data comes in, train the existing model with the new data. I…
Michael
  • 343
  • 3
  • 13
2
votes
0 answers

How to validate data during inference using tfx / tfdv / tensorflow serving?

I'm building a tfx pipeline and using tensorflow serving to serve my model. I save the signature with model.save(...). So far I was able to use the transform layer to transform the feature before prediction with…
John Tartu
  • 41
  • 2
2
votes
1 answer

How to compute additional statistics in Evaluator?

In the TFX Evaluator, on top of the metrics described in TFMA format,I would like to compute statistics relative to the performance of my model on my dataset. Naturally, I would also like a way to get access to these statistics: either through the…
JM Montanier
  • 78
  • 6
2
votes
1 answer

How to make a custom metric available to TFMA/Beam?

I have created a custom Keras metric, similar to the demo implementation below: import tensorflow as tf class MyMetric(tf.keras.metrics.Mean): def __init__(self, name='my_metric', dtype=None): super(MyMetric, self).__init__(name=name,…
neurix
  • 4,126
  • 6
  • 46
  • 71
2
votes
1 answer

Why isn't SchemaGen supported in tfdv.display_schema()?

Regarding TFX' tensorflow-data-validation, I'm trying to understand when I should use *Gen components vs. using TFDV provided methods. Specifically, what's confusing me is that I have this as my ExampleGen: output = example_gen_pb2.Output( …
Lorin S.
  • 754
  • 8
  • 29
2
votes
1 answer

Orchestrating TFX Pipelines with Kubeflow locally

Hey I am working on a package which generates a TFX Pipelines for training GPT-2 (see https://github.com/steven-mi/tfx-gpt2). I was wondering how I am able to deploy my pipeline to Kubeflow locally. Is there any in depth guide for doing so?
stmi
  • 21
  • 2
2
votes
1 answer

Converting TFRecords and tf.Examples to commonly used data types

I am learning to create TensorFlow Extended pipelines and find them greatly useful. However, I have yet to figure out how to debug and test (tabular) data passing through these pipelines. I am aware TensorFlow uses TFRecords/tf.Examples, which are…
Henry
  • 727
  • 1
  • 6
  • 25
2
votes
1 answer

TFX Example_Gen Produce RuntimeError

I've been getting the following error when I run context.run(example_gen) , and I like to understand what does it mean and how can I avoid getting the error. Please advice and thanks in advance! Error: RuntimeError: Files in same split…
LLTeng
  • 385
  • 3
  • 4
  • 15
2
votes
1 answer

What is the difference between implementation of 'drift_comparator' and 'skew_comparator' available in Tensorflow Data Validation?

Need help some in understanding the difference between the underlying implementation of skew and drift comparator in TFDV.
arnab_0017
  • 41
  • 1
  • 1
  • 4
2
votes
0 answers

Failed to convert object of type to Tensor

I'm trying to build a ML production pipeline using TFX, and currently working on Trainer module. I need to implement modelling in a separate file. Here is part of pipeline, which handle training: trainer = Trainer( module_file=module_file, …
Petro Franchuk
  • 53
  • 4
2
votes
1 answer

TensorFlow Serving Cluster Architecture

Folks, I am writing an application which will produce recommendations based on ML model call. The application will have different models, some of them should be called in sequence. A data scientist should be able, to upload a model in the system.…
Andras Gyacsok
  • 61
  • 7
1 2
3
…
12 13