How to merge Pre-post processing of ML model into ONNX format

Question

Simply inside the model should pre-processing be done; for inference, the user should only give the image path. Inside the onnx model, colour conversion and picture resizing will be performed. Please provide suggestions.

    # Preprocessing of ONNX model
    img = cv2.imread(testImage_path)
    img=cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    img = cv2.resize(img, dsize=(180, 180))
    img.resize((1,180, 180, 3))

    # Inference ONNX Model
    ort_sess = ort.InferenceSession('./1/Image_Classific_Retrain_14022022_RGBInput.onnx')
    input_name = ort_sess.get_inputs()[0].name
    outputs = ort_sess.run(None, {input_name: img.tolist()})

score 2 · Answer 1 · answered Apr 02 '22 at 18:47

Including the preprocessing in the model is possible, but not to the extend that you are asking for.
For instance, what is impossible is to just give the model a filepath, and expect it to read data from there. A generell rule of thumb is that if a operation can be vectorized, it's possible to include in the model. Also, usually in each framework that can be used to make a onnx model, as long as you can express your preprocessing logic entirely with the data structures of that framework (for instance torch tensors in pytorch), it can be included in the model.
To find more detailed information about what is possible in onnx models at all, you could check out the Operator list.

Now, from that we can say that preprocessing using open-cv, as you seem to be doing in your question, won't be possible within the model. However, if you can express the same logic in the framework the onnx model was made with, it's still possible to do resizing and color conversion.
For instance, using pytorch as an example, you could incorporate the torchvision.transforms.Resize model into your's for resizing, and the color conversion in terms of tensors, is also just a permutation of the channels.
All of these components can be included in the model, and still be exported.
Note that you would still need to send the image data to the model. For that I would recommend Pillow, as it works very well with pytorch as well as numpy arrays.

Still, that is just an example of how to approach this in pytorch. Very similar techniques can be used in different frameworks, but for that you'll have figure out what strategies / technologies work best for you.

How to merge Pre-post processing of ML model into ONNX format

1 Answers1