Changing LD_LIBRARY_PATH at runtime for ctypes

Even if you give a fully qualified path to CDLL or cdll.LoadLibrary(), you may still need to set LD_LIBRARY_PATH before invoking Python. If the shared library you load explicitly refers to another shared library and no "rpath" is set in the .so for that library, then it won't be found, even if it has already been loaded. An rpath in a library specifies a search path to be used to search for other libraries needed by that library

For example, I have a case of a set of interdependent third-party libraries not produced by me. b.so references a.so. Even if I load a.so in advance:

ctypes.cdll.LoadLibrary('/abs/path/to/a.so')
ctypes.cdll.LoadLibrary('/abs/path/to/b.so')

I get an error on the second load, because b.so refers to simply 'a.so', without an rpath, and so b.so doesn't know that's the correct a.so. So I have to set LD_LIBRARY_PATH in advance to include '/abs/path/to'.

To avoid having to set LD_LIBRARY_PATH, you modify the rpath entry in the .so files. On Linux, there are two utilities I found that do this: chrpath, and patchelf. chrpath is available from the Ubuntu repositories. It cannot change rpath on .so's that never had one. patchelf is more flexible.

By the time a program such as Python is running, the dynamic loader (ld.so.1 or something similar) has already read LD_LIBRARY_PATH and won't notice any changes thereafter. So, unless the Python software itself evaluates LD_LIBRARY_PATH and uses it to build the possible path name of the library for dlopen() or an equivalent function to use, setting the variable in the script will have no effect.

Given that you say it doesn't work, it seems plausible to suppose that Python does not build and try all the possible library names; it probably relies on LD_LIBRARY_PATH alone.

CDLL can be passed a fully qualified path name, so for example I am using the following in one of my scripts where the .so is in the same directory as the python script.

import os
path = os.path.dirname(os.path.realpath(__file__))
dll = CDLL("%s/iface.so"%path)

In your case the following should suffice.

from ctypes import *
lib = CDLL("/home/starlon/Projects/pyCFA635/lib/libevaluator.so")

Compile your binary with a rpath relative to the current working directory like:

gcc -shared -o yourbinary.so yoursource.c otherbinary.so \
    -Wl,-rpath='.',-rpath='./another/relative/rpath' -fpic

Then, you are able to change the working directory in python at runtime with:

import os
os.chdir('/path/to/your/binaries')

Like this, the loader also finds other dynamic libraries like otherbinary.so