1

I'm trying to embed python in my C++ application.

For convenience I would like to place all modules (Lib folder) in a single zip file.

However, some modules rely on *pyd and .dll files that can not be loaded from memory (since windows cannot load dlls from memory).

There is the ZipExtImporter package* that seems to circumvent this issue by loading in memory the natives components via the memory module (credit Joachim Bauch).

(see also : load pyd files from a zip from embedded python )

However, I do not want to load from memory, as on a 32Bit computer the virtual address space of the calling process may be limited.

Then I'm wondering if there is a way to redirect call to those natives components to a folder that contains all *.dll/*.pyd.

The idea is to extract all .pyd/dlls before to zip the Lib folder, to put them into a common folder and to redirect import statement to this external folder.

I believe this can be done via sys.meta_path but I haven't succeed it.

*this package is not maintained anymore.

So my question is :

How can I redirect call to '.pyd/.dll' files that I moved outside of the zip file (without modifying the source code of modules) ?

Edit : To give you a sense of what I'm trying to do, see below my code that is supposed to catch an import statment to a specific pyd file and to redirect it to another location. At this stage I only tried on a non-zipped folder but it doesn't work...

# just test for a single pyd file that I removed from a non-zipped folder
class CustomImporter(object):

   virtual_name = 'pandas._libs.tslib' # a pyd file

   def find_module(self, fullname, path=None):

      if fullname == self.virtual_name:
         return self

      return None

   def load_module(self, fullname):

      if fullname in sys.modules:
            return sys.modules[fullname]

      if fullname != self.virtual_name:
         raise ImportError(fullname)
      try:
          path = r"C:\.....\tslib.cp35-win_amd64.pyd"
          module = imp.load_dynamic("tslib",path )
          module.__file__ = path
          module.__loader__ = self
          module.__name__ = self.virtual_name
          sys.modules[self.virtual_name] = module
      except BaseException as e:
          raise ImportError(fullname + str(e))

      return module

and it is attached as follow : 

sys.meta_path.append(CustomImporter())
Malick
  • 6,252
  • 2
  • 46
  • 59
  • Other than frobbing `sys.path`? – Ignacio Vazquez-Abrams Oct 23 '17 at 22:22
  • @IgnacioVazquez-Abrams I'm interested by all the ways, but I have added a small example to be more understandable – Malick Oct 23 '17 at 22:42
  • @IgnacioVazquez-Abrams `*.pyd/*.dll` are sometimes expected to be in a specific location (ex : `from . import mypyd` or `import .mypyd` ..) and then I don't think we can play with `sys.path` to handle these cases. – Malick Oct 23 '17 at 22:49

0 Answers0