Importing pandas module stops my script from working

Question

I am using child_process model to send data to a python script, where I preform some calculations with pandas, and then send it back to a node.js script: index.js. The problem is that when I import the pandas module to the python script, it stops the script from returning data to my index.js script (I dont know why). However, if I simply dont import the pandas module, i get data returned from the python script.

This is how i send the data to the python script from index.js:

const spawn = require('child_process').spawn
let result = ''

const pythonProcess = spawn('python',['./rl-commands/t1.py', a, b]); #a, b are arguments that I send through. In this case they are just som integers (2 and 2)

Then I process the data in the python script like this (keep in mind i am not actually using pandas here, because I am just trying to make the connection between the two scripts work first, but pandas is still necessary):

import sys
import json
import random
import numpy as np
import pandas as pd

a = sys.argv[1]
b = sys.argv[2]

print(int(a) + int(b))
sys.stdout.flush()

Finally I retrieve the code that i processed in python to my index.js script:

pythonProcess.stdout.on('data', (data) => {
    result += data.toString()
});
pythonProcess.on('close', function (code) {
    console.log("RES: ", result);
});

!!! As explained earlier, this does not work. But if I comment out the pandas import from the python script, it works:

import sys
import json
import random
import numpy as np
#import pandas as pd

a = sys.argv[1]
b = sys.argv[2]

print(int(a) + int(b))
sys.stdout.flush()

I dont understand how removing import pandas as pd makes the script run?

first step is to see if it's throwing an error that you aren't seeing because you are running it from javascript, do: `try: import pandas; except Exception as e: print("GOT ERROR:",e)` and see if that gives you any output? Or even better see if there is a way to read the error output from the javascript file. — Tadhg McDonald-Jensen, Jul 08 '20 at 17:18
@TadhgMcDonald-Jensen , I did this and got: ``GOT ERROR: No module named 'pandas'``. However this is not true. I have the newest version of pandas installed in the same place as all the other modules. And pandas always works with my other python scripts. — ulisman1, Jul 08 '20 at 17:33
ok well then the first question is are you sure it is running the version of python you are expecting it to run? if the script just does `import sys; print(sys.version); import pandas` does it give the same version but in one place (terminal) it successfully imports but in another (javascript) it fails? — Tadhg McDonald-Jensen, Jul 08 '20 at 17:38
it might be as simple as using `spawn('python3',...)` instead of `'python'`. — Tadhg McDonald-Jensen, Jul 08 '20 at 17:40
@TadhgMcDonald-Jensen Well, I have selected the newest python3 verion in my interpreter. But intrestingly, if I use ``import sys; print(sys.version); import pandas`` as you said, it prints ``v 3.8.2`` in python terminal, but ``3.7.6`` in node.js terminal. — ulisman1, Jul 08 '20 at 18:07
ok well then to revise last statement: it might be as simple as using `'python3.8'` instead of `'python'`. did you set it to 3.8 in an IDE or does the "python" command run in bash give a different version than the same command in nodejs? — Tadhg McDonald-Jensen, Jul 08 '20 at 18:12
@TadhgMcDonald-Jensen Yes! Changing to ``'python3.8'`` worked. Thank you! — ulisman1, Jul 08 '20 at 18:35

Tadhg McDonald-Jensen · Accepted Answer · 2020-07-08T18:14:01.997

Since it fails to run there is almost certainly some data written to stderr containing the error message so I would highly recommend you do something like this in your javascript code:

pythonProcess.stderr.on('data', (data) => {
    console.error(data)
});

then it will output what the error was in the python script to your javascript terminal.

Once you know what is going wrong (almost certainly that the pandas module isn't installed in this specific instance) it is very possible that python command isn't referring to the version of python you are expecting it to. try running:

spawn('python',["--version"])

or similar to confirm it is the right version, it's possible you need to use 'python3' or 'python3.6' etc. as the executable to specify the specific version to use instead of just 'python'

Importing pandas module stops my script from working

1 Answers1