Send data from Java to Python and then return the response

Question

I have a Java program which gets data from a database in the form of a list of objects and an object just has some primitive types associated to it like strings and ints.

Now I need to pass the data from Java to a Python program to do some calculations and then return the calculated output to the Java program.

My Python program has the following imports

import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf
import numpy as np
import scipy.misc as sp
from scipy.optimize import minimize
from scipy.stats import poisson

Does anyone have any ideas for how I could achieve this? I looked at Jython but saw that it isn't compatible with NumPy and others but there is a compatibility layer available called JyNI however I can't find many examples to getting a working solution.

EDIT: From my research, there doesn't seem to be a suitable library which will handle communication between Java and Python but would it be possible to the Java program to start a Python script which looks for say a JSON file containing the data it needs for processing and then prints an output that Java can pick up?

Answer 1

I am of the opinion, that you stand to have several choices; my personal favourites are as follows:

• I'm not too sure why you are trying to speak between Python and Java, so I'd like to make the first point of staying away from such things, if at all possible. I understand that there are some things that simply cannot be done in a language, certain libraries that are hard to find/emulate in another language, etc.; but, one will find it often simply to be less of a hassle to work in one language to complete the whole of the task. That being said, if you still feel that you wish you use both languages, I will not stop you, since you probably have a good reason.
• If you have chosen to continue, then, let me provide you with a quick few ideas that might help in circumventing this issue. Albeit it's true that Jython is incompatible with NumPy, there are other libraries that may accomplish your task similarly. A few examples: DataMelt (a JVM language family friendly way to do graphing, but also has very strong math capabilities); EJML (a great thing for working with big matrices, and it's open source :D); and ND4J (a Java library very similar to NumPy).
• Finally, if you find difficulty in the above, or you simply choose to not use it, there is always the option of abstraction. I say this, in the sense that one can always choose to move around which tasks are done where: this is applicable to you, in that you could move some of the calculations and processes around, so that Java does more of the work, minimizing the things you actually would need to send, to a few simple structures that could be conveyed either by a scripting language in the JVM family, (like Rhino, a JavaScript derivative, etc.), or by using something like Jacl to have LAN connection through TCP, and to broadcast one program's results to another, through I/O methods.

There are probably many other little workarounds and solutions that people have come up with in the past (native embeddings, higher level control, etc.), but these are my best offers to you, as they cause the least trouble (as far as I am aware). I hope this answers your question, and best of luck to you.

A short list of maths libraries useful in Java.