2

I want to build a training pipeline using TFX, and eventually to reuse my data transformations to make inference requests to TensorFlow-Serving, which TFX is supposedly able to do. The TFX examples I found all seem to build a batch training pipeline and eventually push the model in TensorFlow-Serving, but they don't address the inference part, which must be a streaming pipeline for latency reasons. I could probably write my own tool to make the request, but it seems a waste not to reuse my Transform component for the inference part.

I have run locally the examples installed in dags by the TFX examples setup script. The airflow UI makes it clear that those are batch pipelines.

Vincent Bernardi
  • 55
  • 4

1 Answers1

1

TFX allows you to define your transform logic inside the training pipeline, and save the logic as part of the resulting model graph, so that your saved model will include both the transformations and the usual model, and tf serving will be able to accept request in pre-transform data format, and do both the proper transformations and model inference without any additional work. Therefore by design TFX is not involved in inference.

Happy Gene
  • 502
  • 3
  • 7
  • As a follow-up, is there an easy way to plug my custom SerDe code inside the TFX graph so that TF Serving is able to deserialize the data before transforming it, or should I do that in a separate layer? – Vincent Bernardi Nov 29 '19 at 09:52
  • I don't think so. TFX requires you to use tf.* supported transformations, so that they can guarantee the transformations can be serialized and deserialized properly. See this link for the API: https://www.tensorflow.org/tfx/transform/api_docs/python/tft. The API does include many commonly used transformations, and you have to port your custom SerDe code to its provided calls. – Happy Gene Nov 30 '19 at 15:30
  • that said, one of the supported call is `apply_pyfunc`, which allows more flexibility. https://www.tensorflow.org/tfx/transform/api_docs/python/tft/apply_pyfunc – Happy Gene Nov 30 '19 at 15:42