2

I want to pass a dataframe from python to a julia function, perform some calculations and then pass a dataframe back to python.

From How to convert a Python pandas into a Julia DataFrame (using PyJulia) and back to Python Pandas by setting the variables in the Julia environment we can do that which works:

import pandas as pd
from julia import Main

data = {
  "temperature": [420, 380, 390],
  "sun": [50, 40, 45]
}

df = pd.DataFrame(data)
print(df) #print initial dataframe

Main.eval('include("compute_milp.jl")') #source julia script
def call_julia():
    Main.df = df #set variable in julia env

    return Main.eval(f'compute_milp()') #execute julia function from julia script

df1 = call_julia() #pass dataframe, then return
print(df1) #print dataframe like initial

In the same folder, we have the dummy script compute_milp.jl

function compute_milp()
  return df
  end

This previous code is working well to return dataframe. But, without converting to array numpy and global variable, I would like something more clean and functional like:

Main.eval('include("compute_milp.jl")') #source julia script
def call_julia(df_arg): # ADD df
    #Main.df = df #SUPPRESS THIS SET

    return Main.eval(f'compute_milp({df_arg})') #execute julia function from julia script

df1 = call_julia(df) #pass dataframe, then return, ADD ARGUMENT
print(df1)

I get these errors with the functional type code:

Traceback (most recent call last):
  File "c:\Users\\python_julia\stack.py", line 19, in <module>
    df1 = call_julia(df) #pass dataframe, then return
  File "c:\Users\python_julia\stack.py", line 17, in call_julia
    return Main.eval(f'compute_milp({df_arg})') #execute julia function from julia script
  File "C:\Users\Anaconda3\envs\optim\lib\site-packages\julia\core.py", line 627, in eval
    ans = self._call(src)
  File "C:\Users\Anaconda3\envs\optim\lib\site-packages\julia\core.py", line 555, in _call
    self.check_exception(src)
  File "C:\Users\Anaconda3\envs\optim\lib\site-packages\julia\core.py", line 609, in check_exception
    raise JuliaError(u'Exception \'{}\' occurred while calling julia code:\n{}'
julia.core.JuliaError: Exception 'syntax: missing comma or ) in argument list' occurred while calling julia code:
compute_milp(   temperature  sun
0          420   50
1          380   40
2          390   45)

Any idea please?

Adri1
  • 23
  • 6
  • Slightly off-topic perhaps: if your python codebase is not too large, then I would strongly recommend just doing the whole thing in Julia instead of jostling with this two-language problem. It will be much easier. – loonatick Apr 26 '23 at 11:32
  • I have code that uses pytorch to make weather predictions and JuMP code to manage energy generation. I'm afraid I'll spend more time translating everything than just switching dataframes from one language to another :-) – Adri1 Apr 26 '23 at 12:52
  • Ah all right. So, does my answer look like what you might need? – loonatick Apr 26 '23 at 14:38

1 Answers1

1

I find it hard to figure out what exactly you mean by functional. So, I am assuming that you want to avoid global variables.

Starting off by summarizing what exactly is going wrong:

The last line of the error is missing comma or ) in argument list, and you can see afterwards how exactly it is trying to call the function compute_milp.

This is happening because the string formating in Main.eval(f'compute_milp({df_arg})') is interpolating the argument string of eval with the string representation of the dataframe, i.e. it is as if you are running a julia script like

compute_milp( temperature sun
0          420   50
1          380   40
2          390   45)

From what I understand, df has to be in the Julia environment to be callable from Julia code. So, you ought to uncomment the assignment to Main.df and settle for a compromise like so.

def call_julia(JlEnv, df_arg):
    JlEnv.df = df_arg  # pass dataframe to Julia environment
    return JlEnv.eval("compute_milp(df)") # without string formating

df1 = call_julia(Main, df)
print(df1)

Since we're trying to be "functional", I would pass the Julia environment explicitly as well.

loonatick
  • 648
  • 1
  • 7
  • 15