0

According to my research, TFX does not provide support for custom feature transformation within the TFX Pipeline as my custom python functions cannot be serialized inside the graph. Therefore making said functions unavailable at serving/inference time.

One alternative I found was to extend TensorFlow Serving by creating a custom Servable (https://www.tensorflow.org/tfx/serving/custom_servable) , but I haven't found any concrete examples of adding feature transformation code.

How can I wrap feature transformations around a SavedModel?

MachineLearner
  • 21
  • 2

1 Answers1

0

There are different kind of solutions:

Option A: You can use TensorFlow Serving to serve your trained and saved model. Then you can preprocess and prepare the data for a request. After that you can use the tensorflow-serving-api package to send a PredictRequest via gRPC to the previous started service:

import tensorflow as tf

from tensorflow_serving.apis.prediction_service_pb2_grpc import (
    PredictionServiceStub
)
from tensorflow_serving.apis.predict_pb2 import (
    PredictRequest,
    PredictResponse,
)


values: List[float] = # ... YOUR PREPROCESSED DATA ...

host, port = '0.0.0.0', 8500
channel = grpc.insecure_channel(f'{host}:{port}')
stub = PredictionServiceStub(channel)

request = PredictRequest()
request.model_spec.name = 'cifar10'
request.model_spec.signature_name = 'serving_default'
tensor_proto = tf.make_tensor_proto(
    values=values,
    shape=(1, 32, 32, 3),
    dtype=float
)
request.inputs['input'].CopyFrom(tensor_proto)
response: PredictResponse = stub.Predict(request)
result = response.outputs['output'].float_val

Option B: You could try to use LambdaLayers to embed preprocessing steps to a model. I've never tried it out, but you should be careful if you want to use custom imports and dependencies, because if we use the regular load_model method to load a saved model we have to apply the required custom_objects.

Darius
  • 10,762
  • 2
  • 29
  • 50
  • Thanks for the response. Currently im my solution resembles Option A, having the data pass through two "pipelines". The first "pipeline" is a script that applies many python functions that transform raw features (based on business logic requirements) later handing the results over to the tfx pipeline. Approaches like LambdaLayers or tf.pyfunc are not ideal as they are not serializable into the Tensorflow graph and therefore not portable. The desired solution would be a single pipeline that doesn't rely on having python available at inference/serving. Any suggestions? – MachineLearner Feb 17 '21 at 13:52