Shell scripts, python, ld_library

Question

I've got a program that calls a shell script from python. In the shell script, I add to the LD_LIBRARY_PATH like so, pointing at the .so's from mdbtools.

NEWPATH=${PWD}"/mdbtools/usr/lib/x86_64-linux-gnu"
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$NEWPATH

I'm pretty sure this part of the code is correct. I then call one of the programs from the .so, "mdb-export" like so, using variables from the input. :

mdb-export ${1} Athlete > "${2}athlete.csv"

Here's the confusing part. When I call the script from the commandline

./buildscvs abcd.mdb 1234

it works perfectly. The csvs are populated as expected. However, when I call the script from python:

    try:
        p = Popen([cmd, filename, prepend],  stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True).communicate()[0]
        print "p", p
    except Exception as e:
        print str(e)

cmd, filename, and prepend are as follows

filename = /app/djangoFiles/meetPlanner/2011ROXBURY.mdb
cmd = /app/djangoFiles/meetPlanner/buildcsvs.sh
prepend = 824365891

The csvs come out blank. This seems consistent with what happens when you run a command with an unknown program name, like if I ran

abcd 1234.mdb > test.csv

Does anyone have any idea what's going on? I'd very much appreciate any help you can give me

Thank you!

EDIT

After changing to subprocess.check_call()

as suggested below, I now get the following error message:

/mdb-export: error while loading shared libraries: libmdb.so.2: cannot open shared object file: No such file or directory

Why are you redirecting stdin and stdout to a pipe when you aren't using them? Just use `subprocess.check_call()` instead and any stdout and stderr should be printed, which will make debugging this vastly easier. — Michael Hoffman, Jul 05 '12 at 03:49
That helped -- thank you! Now I get this `/app/djangoFiles/meetPlanner/mdb-export: error while loading shared libraries: libmdb.so.2: cannot open shared object file: No such file or directory`. This doesn't come up when I run it from the terminal. Any ideas? — mythander889, Jul 05 '12 at 04:07

Igor Chubin · Answer 1 · 2012-07-05T04:53:19.890

2

You must check not only stdout but stderr also.

(stdout, stderr) = Popen([cmd, filename, prepend],  stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True).communicate()
if stderr != "":
    print stderr

Also you write that ou set LD_LIBRARY_PATH, but where? In the script or in the python program? If in the python program you must pass the environment hash in the Popen call, but you don't do this.

The error that you get

/app/djangoFiles/meetPlanner/mdb-export: error while loading shared libraries: libmdb.so.2: cannot open shared object file: No such file or directory

means exactly that that LD_LIBRARY_PATH is set incorrectly (or is not set at all).

update

Ok, now we know that you set LD_LIBRARY_PATH in the script. Don't forget to export it.

export LD_LIBRARY_PATH

edited Jul 05 '12 at 04:53

answered Jul 05 '12 at 04:08

Igor Chubin

61,765
13
122
144

I set the LD_LIBRARY_PATH in the script itself – mythander889 Jul 05 '12 at 04:10
1

do you export it? I mean in the script? it is possible that in the shell the variable is already exported ant that is why the script works, but in python script not yet – Igor Chubin Jul 05 '12 at 04:13
That makes sense. Do you have any suggestions for how to debug how to set it correctly? It works when I call it from command line, but not from the python program – mythander889 Jul 05 '12 at 04:14
I don't export it, but would be happy to try. What do I need to do to export it? – mythander889 Jul 05 '12 at 04:15
2

Just add `export LD_LIBRARY_PATH` before the call of `mdb-export` in the script (to check the correctness of the idea). – Igor Chubin Jul 05 '12 at 04:16
Got it! Thank you -- my path was off – mythander889 Jul 05 '12 at 04:38

Shell scripts, python, ld_library

1 Answers1