Change current process environment's LD_LIBRARY_PATH

Question

Is it possible to change environment variables of current process?

More specifically in a python script I want to change LD_LIBRARY_PATH so that on import of a module 'x' which depends on some xyz.so, xyz.so is taken from my given path in LD_LIBRARY_PATH

is there any other way to dynamically change path from where library is loaded?

Edit: I think I need to mention that I have already tried thing like os.environ["LD_LIBRARY_PATH"] = mypath os.putenv('LD_LIBRARY_PATH', mypath)

but these modify the env. for spawned sub-process, not the current process, and module loading doesn't consider the new LD_LIBRARY_PATH

Edit2, so question is can we change environment or something so the library loader sees it and loads from there?

Isn't this a duplicate of http://stackoverflow.com/questions/856116/changing-ldlibrarypath-at-runtime-for-ctypes? You aren't actually asking how to change the environment, but rather how to change where python loads libraries from. — Bryan Oakley, Jul 24 '09 at 15:55
ok! i will say this is duplicate, but no it is not abt python it about changing environment, e.g. in a c app which loads a dynamic lib A, can we change environ so that A is loaded from our path of choice — Anurag Uniyal, Jul 24 '09 at 17:32

score 39 · Accepted Answer · edited May 23 '17 at 12:00

39

The reason

os.environ["LD_LIBRARY_PATH"] = ...

doesn't work is simple: this environment variable controls behavior of the dynamic loader (ld-linux.so.2 on Linux, ld.so.1 on Solaris), but the loader only looks at LD_LIBRARY_PATH once at process startup. Changing the value of LD_LIBRARY_PATH in the current process after that point has no effect (just as the answer to this question says).

You do have some options:

A. If you know that you are going to need xyz.so from /some/path, and control the execution of python script from the start, then simply set LD_LIBRARY_PATH to your liking (after checking that it is not already so set), and re-execute yourself. This is what Java does.

B. You can import /some/path/xyz.so via its absolute path before importing x.so. When you then import x.so, the loader will discover that it has already loaded xyz.so, and will use the already loaded module instead of searching for it again.

C. If you build x.so yourself, you can add -Wl,-rpath=/some/path to its link line, and then importing x.so will cause the loader to look for dependent modules in /some/path.

edited May 23 '17 at 12:00

Community

1
1

answered Jul 27 '09 at 02:33

Employed Russian

199,314
34
295
362

B, looks like an option I can use – Anurag Uniyal Jul 27 '09 at 05:58
How would option B be implemented? I think the OP implied that `/some/path/xyz.so` is a shared library that a Python module depends on; `xyz.so` does not contain a Python module itself. I have a similar situation and don't know of any way to import the library in the way you describe. – Jason R Apr 13 '12 at 04:33
2

@JasonR Option B could be implemented using `import dl; dl.open("/some/path/xyz.so")` – Employed Russian Apr 13 '12 at 06:28
4

Outstanding; thanks. Turns out that the `dl` module is deprecated as of Python 2.6, but you can do the same with `import ctypes; ctypes.cdll.LoadLibrary("/some/path/xyz.so")`. – Jason R Apr 13 '12 at 12:43
@JasonR Did option B work out for you? I've tried using importing the full path to the shared lib with ctypes right before importing the python module which depends on it.. but no luck.. imp.load_module() still fails – Gregory Jan 26 '13 at 18:24
1

@Gregory: Yes it did. I used the syntax that I indicated in my comment above, using `ctypes`, and it worked well for me. It is possible that your Python module also depends upon some other shared libraries that haven't been loaded properly. – Jason R Jan 27 '13 at 16:24
@JasonR It depends on a bunch, but they either were linked with full paths (standard libs) or have an rpath. The one that doesnt have an rpath always fails when imported into python.. BTW. It's a SWIG python wrapper around a shared library and SWIG suggests to use rpath/LD_LIBRARY_PATH which is exactly what I dont want to do because it's not portable/moveable/copyable – Gregory Jan 29 '13 at 23:34

score 16 · Answer 2 · edited Nov 26 '14 at 17:08

16

Based on the answer from Employed Russian, this is what works for me

oracle_libs = os.environ['ORACLE_HOME']+"/lib/"
rerun = True

if not 'LD_LIBRARY_PATH' in os.environ:
  os.environ['LD_LIBRARY_PATH'] = ":"+oracle_libs
elif not oracle_libs in os.environ.get('LD_LIBRARY_PATH'):
  os.environ['LD_LIBRARY_PATH'] += ":"+oracle_libs
else:
  rerun = False

if rerun:
  os.execve(os.path.realpath(__file__), sys.argv, os.environ)

edited Nov 26 '14 at 17:08

tssch

734
8
25

answered May 13 '13 at 08:10

cristi

2,019
1
22
31

1

The execve trick is *exactly* what I needed, and it works, have an upvote! – Kenneth Hoste Feb 15 '16 at 10:10

score 0 · Answer 3 · answered Jul 24 '09 at 16:55

In my experience trying to change the way the loader works for a running Python is very tricky; probably OS/version dependent; may not work. One work-around that might help in some circumstances is to launch a sub-process that changes the environment parameter using a shell script and then launch a new Python using the shell.

score 0 · Answer 4 · answered Aug 19 '22 at 17:52

The below code is to set the LD_LIBRARY_PATH or any other environment variable paths that is required by the import modules.

if os.getenv('LD_LIBRARY_PATH')==None: 
    os.environ['LD_LIBRARY_PATH']='<PATH>'
    try:
        sys.stdout.flush()
        os.execl(sys.executable,sys.executable, *sys.argv)
    except OSError as e:
        print(e)
elif <path> not in os.getenv('LD_LIBRARY_PATH'):
    os.environ['LD_LIBRARY_PATH'] = ':'.join([os.getenv('LD_LIBRARY_PATH'),'<PATH>'])
    try:
        sys.stdout.flush()
        os.execl(sys.executable,sys.executable, *sys.argv)
    except OSError as e:
        print(e)

# import X

The function os.execl will replace the current process. In UNIX a new executable will be loaded into the current process. By having this code before the import of the 'X' module, now it will be looking for the files in the new path that was set.

Change current process environment's LD_LIBRARY_PATH

5 Answers5

Linked

Related