Generating predictions with a Tensorflow Extended pipeline

Question

I'd like to use a saved TFX pipeline to generate predictions using a saved TFX pipeline object, so something like this:

model = load_tfx_model("path/to/artifact")
model.predict(new_data)

Importantly, I'd like to apply a pre-processing pipeline to the inputs before passing them to the model for inference (similar to sklearn pipelines).

It appears that the BulkInferrer can generate predictions and I've also found a REST example but what I cannot figure out is whether either of these options is actually going to apply the TFX Transform step (preprocessing_fn) on the new data. Of the examples of preprocessing_fn that I've seen, like this one, the response variable is also modified in the transform but since it will not be available during prediction, I suspect this this function is only used for training. Would anyone be able to shed light on this?

Many thanks!

Answer 1

The BulkInferrer TFX component performs batch inference on unlabeled data. It is typically deployed after an Evaluator component to perform inference with a validated model, or after a Trainer component to directly perform inference on exported model. For more info you can refer BulkInferrer API.

Edit:

Yes, BulkInferrer will apply preprocessing_fn on the new data. The Transform component consumes data as tf.Examples, performs feature engineering(applies preprocessing_fn) and returns a SavedModel to a Trainer component. This trained model from Trainer component is used by BulkInferrer to perform batch inference on unlabeled data.

TensorFlow Transform builds transformations into the TensorFlow graph for your model so the same transformations are performed at training and inference time. So, the preprocessing_fn is part of you SavedModel graph, and will be applied on unlabelled data before making predictions.