Integrating Model with Data Preprocessing steps

Question

It might be a silly thought, but please bear with me and guide me if I am taking the wrong approach. I am working on a Machine learning project whose model will give final output, this output from ML model is to be consumed by another project written and maintained in java.

A typical flow of REST call is as follows:

---

1. The user requests some classification scores at UI.
2. UI system calls Java API.
3. Java API fetches data from the Storage system.
4. then it preprocesses the data and presents it to the Trained model which is completely written, developed and trained in Python.

Problem: While making the ML model we have preprocessing steps written in python, data comes in and gets preprocessed and model training starts after that. After this model is being persisted to the location from where java can use this for real-time prediction.

Thought: Is it possible to wrap the preprocessing steps with the model as one package. so that at the java side the preprocessing can be avoided?

I want all your opinions on this for better handling of the situation. I do not want to write and maintain Feature engineering/Preprocessing steps at two places(Python and Java).

Please guide.

Answer 1

Its possible to wrap up the preprocessing steps model into one package by using TensorFlow for Java which is a Machine Learning Model that can supplement the python model. Considering you have wired your Java API to the UI and system storage, no need of rewriting them or you can reuse thus you can save on time.

• Structure your project using maven project

• Import relevant libraries for instance Spring RESTful webservice, Spring JPA Data, Spring Thymeleaf / Angular (UI) and tensor flow library for ML model

    <dependency>
       <groupId>org.tensorflow</groupId>
       <artifactId>tensorflow</artifactId>
       <version>1.12.0</version>
    </dependency>

I hope this works for you.