pygst + pyinstaller under OSX

Question

I'm currently trying to bundle this dead simple python script (player.py):

#!/usr/bin/env python

import sys, os
import gobject, glib
import pygst
pygst.require("0.10")
import gst

class Player(object):
    def __init__(self):
        self.player = gst.element_factory_make("playbin2", "player")
        fakesink = gst.element_factory_make("fakesink", "fakesink")
        self.player.set_property("video-sink", fakesink)
        bus = self.player.get_bus()
        bus.add_signal_watch()
        bus.connect("message", self.on_message)

    def play(self, url):
        self.player.set_state(gst.STATE_NULL)
        self.player.set_property("uri", url)
        self.player.set_state(gst.STATE_PLAYING)

    def on_message(self, bus, message):
        t = message.type
        if t == gst.MESSAGE_EOS:
            global loop
            loop.quit()
        elif t == gst.MESSAGE_ERROR:
            self.player.set_state(gst.STATE_NULL)
            err, debug = message.parse_error()
            print "Error: %s" % err, debug

p = Player()
p.play("file:///path/to/something.mp3")
gobject.threads_init()
loop = glib.MainLoop()
loop.run()

into an OSX (my machine runs Mountain Lion) application using pyinstaller (2.1.0-dev).

My aim is to create a .app bundle that I can easily distribute. I could also ask the final user to install GStreamer SDK, even though a self-contained application would be my primary goal.

The spec file follows (player.spec):

# -*- mode: python -*-
import pygst
pygst.require('0.10')

a = Analysis(['player.py'],
             pathex=['/Users/mymy/devel/t/simple'],
             hiddenimports=[],
             hookspath=None,
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='player',
          debug=False,
          strip=None,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='player')

I attempted two strategies, so far:

macports: python (2.7.2), pygst (0.10), gst-plugins-*
system python, GStreamer 0.10 SDK (Framework)

In both cases I can successfully run the script.

When I attempt to run the bundled executable, though, I get the following:

(Macports)

Showing a partial GST debug log:

~/devel/t/simple/dist/player > GST_DEBUG=4 ./player
[..]
0:00:00.113260000  8313    0x1001b1a00 DEBUG           GST_REGISTRY gstregistrychunks.c:573:gboolean gst_registry_chunks_load_feature(GstRegistry *, gchar **, gchar *, GstPlugin *): Plugin 'playback' feature 'playbin2' typename : 'GstElementFactory'
0:00:00.113286000  8313    0x1001b1a00 DEBUG           GST_REGISTRY gstregistrychunks.c:621:gboolean gst_registry_chunks_load_feature(GstRegistry *, gchar **, gchar *, GstPlugin *): Element factory : 'Player Bin 2' with npadtemplates=0
0:00:00.113300000  8313    0x1001b1a00 DEBUG           GST_REGISTRY gstregistrychunks.c:649:gboolean gst_registry_chunks_load_feature(GstRegistry *, gchar **, gchar *, GstPlugin *): Reading 2 Interfaces at address 0x101971191
0:00:00.113318000  8313    0x1001b1a00 DEBUG           GST_REGISTRY gstregistry.c:558:gboolean gst_registry_add_feature(GstRegistry *, GstPluginFeature *):<registry0> adding feature 0x10097da20 (playbin2)
0:00:00.113332000  8313    0x1001b1a00 DEBUG        GST_REFCOUNTING gstobject.c:844:gboolean gst_object_set_parent(GstObject *, GstObject *):<playbin2> set parent (ref and sink)
0:00:00.113346000  8313    0x1001b1a00 DEBUG           GST_REGISTRY gstregistrychunks.c:709:gboolean gst_registry_chunks_load_feature(GstRegistry *, gchar **, gchar *, GstPlugin *): Added feature playbin2, plugin 0x100975be0 playback
[..]
0:00:00.242584000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstpluginfeature.c:106:GstPluginFeature *gst_plugin_feature_load(GstPluginFeature *): loading plugin for feature 0x10097da20; 'playbin2'
0:00:00.242620000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstpluginfeature.c:110:GstPluginFeature *gst_plugin_feature_load(GstPluginFeature *): loading plugin playback
0:00:00.242632000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstplugin.c:1293:GstPlugin *gst_plugin_load_by_name(const gchar *): looking up plugin playback in default registry
0:00:00.242662000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstplugin.c:1296:GstPlugin *gst_plugin_load_by_name(const gchar *): loading plugin playback from file /opt/local/lib/gstreamer-0.10/libgstplaybin.so
0:00:00.242677000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstplugin.c:737:GstPlugin *gst_plugin_load_file(const gchar *, GError **): attempt to load plugin "/opt/local/lib/gstreamer-0.10/libgstplaybin.so"
0:00:00.248338000  8297    0x1001b1a00 INFO      GST_PLUGIN_LOADING gstplugin.c:859:GstPlugin *gst_plugin_load_file(const gchar *, GError **): plugin "/opt/local/lib/gstreamer-0.10/libgstplaybin.so" loaded
0:00:00.248374000  8297    0x1001b1a00 DEBUG     GST_PLUGIN_LOADING gstpluginfeature.c:115:GstPluginFeature *gst_plugin_feature_load(GstPluginFeature *): loaded plugin playback
0:00:00.248390000  8297    0x1001b1a00 INFO      GST_PLUGIN_LOADING gstpluginfeature.c:145:GstPluginFeature *gst_plugin_feature_load(GstPluginFeature *): Tried to load plugin containing feature 'playbin2', but feature was not found.
0:00:00.248402000  8297    0x1001b1a00 WARN     GST_ELEMENT_FACTORY gstelementfactory.c:410:GstElement *gst_element_factory_create(GstElementFactory *, const gchar *):<playbin2> loading plugin containing feature player returned NULL!
0:00:00.248412000  8297    0x1001b1a00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:472:GstElement *gst_element_factory_make(const gchar *, const gchar *):<playbin2> couldn't create instance!
Traceback (most recent call last):
  File "<string>", line 33, in <module>
  File "<string>", line 11, in __init__
gst.ElementNotFoundError: playbin2

Getting the same result pre-pending: GST_PLUGIN_PATH=/opt/local/lib/gstreamer-0.10/

I tried to copy the plugins and their dependencies into the dist/player folder, scripting a wild install_name_tool mangling in order to correct the paths of the dylibs, but the result doesn't change either.

(GStreamer SDK)

(PYTHONPATH=/Library/Frameworks/GStreamer.framework/Versions/0.10/lib/python2.7/site-packages/)

~/devel/t/simple/dist/player > ./player 
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
player.py:11: Warning: cannot register existing type `GstObject'
player.py:11: Warning: g_once_init_leave: assertion `result != 0' failed
player.py:11: Warning: gtype.c:2720: You forgot to call g_type_init()

and here it hangs. If I sample the process via Activity Monitor, I get this:

[..]
_wrap_gst_element_factory_make  (in gst._gst.so)
gst_element_factory_make  (in libgstreamer-0.10.0.dylib)
gst_element_factory_create  (in libgstreamer-0.10.0.dylib)
gst_plugin_feature_load  (in libgstreamer-0.10.0.dylib)
gst_plugin_load_by_name  (in libgstreamer-0.10.0.dylib)
gst_plugin_load_file  (in libgstreamer-0.10.0.dylib)
gst_plugin_register_func  (in libgstreamer-0.10.0.dylib)
plugin_init  (in libgstplaybin.so)
gst_play_bin2_plugin_init  (in libgstplaybin.so)
gst_pipeline_get_type  (in libgstreamer-0.10.0.dylib)
gst_bin_get_type  (in libgstreamer-0.10.0.dylib)
gst_child_proxy_get_type  (in libgstreamer-0.10.0.dylib)
gst_object_get_type  (in libgstreamer-0.10.0.dylib)
g_once_init_enter  (in libglib-2.0.0.dylib)
g_cond_wait  (in libglib-2.0.0.dylib)
_pthread_cond_wait  (in libsystem_c.dylib)
__psynch_cvwait  (in libsystem_kernel.dylib)

A hint would be immensely appreciated!

Not an answer by any means, but a comment: Installing the MacPorts way means you will have to wrap up, or require the user to install, not just GStreamer, but every dependency of GStreamer or Python. This is not impossible, but it can be very painful. On the other hand, using system Python to create a completely standalone build (especially one that works on older OS versions than yours) can _also_ be painful. — abarnert, May 18 '13 at 00:16
Also, have you tried asking about GStreamer on the [PyInstaller mailing list](http://www.pyinstaller.org/wiki/WikiStart#MailingList), looking through the hook collection, checking for a relevant bug report, etc.? The odds that someone on SO has used PyInstaller on a Mac to package GStreamer are probably much lower than the odds that someone on the PyInstaller mailing list has… — abarnert, May 18 '13 at 00:18
On MacPorts, following the approach of packages like Kivy, I'd brought in all the dependent libraries. This is partly accomplished by pyinstaller (for what concerns the deps of the python extensions) and I created a set of scripts that inspect the plugins .sos and resolve the dependencies, copying them from macports' lib path to the packaged folder, fixing the dylibs paths (using osxrelocator.py, which comes from gstreamer cerbero toolset). Anyway I don't understand why the plugins can't be loaded on my system, even before copying them. — oxullo, May 18 '13 at 08:38
I checked the hooks collection, in particular pyqt, which brings plugins in (successfully!), and the kivy custom hooks, but I couldn't totally figure out their way. I'll try them again in first stance and eventually follow your suggestion about PyInstaller. Thanks! — oxullo, May 18 '13 at 08:43

score 2 · Accepted Answer · answered May 18 '13 at 21:36

I finally have a working solution for the following setup:

macports (python @2.7.3, py27-gst-python @0.10.22, gstreamer010 @0.10.36)
pyinstaller 2.1-dev (ccb6f3d3d924a0dc2f9e92aa6278c28a2d743d39)
OSX 10.8.3

pyinstaller spec file (player.spec):

# -*- mode: python -*-
import os
import pygst
pygst.require('0.10')

a = Analysis(['rthook.py', 'player.py'],
             pathex=[os.curdir],
             hiddenimports=[],
             hookspath=None,
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='player',
          debug=False,
          strip=None,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='player')

_gst.so hook (hook-gst._gst.py):

I created a very primitive hook for gst. I tried to put it inside a local directory, referencing it via the hookspath parameter of the Analysis object, but I couldn't figure out why pyinstaller ignored it. Therefore I moved it to the /path/to/pyinstaller/PyInstaller/hooks folder:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os

GST_PLUGINS = '/opt/local/lib/gstreamer-0.10/'

def hook(mod):
    for f in [so for so in os.listdir(GST_PLUGINS) if so[-3:].lower() == '.so']:
        mod.binaries.append((os.path.join('gst-plugins', f),
                os.path.join(GST_PLUGINS, f),
                'BINARY'))

    return mod

pyinstaller takes care of computing the dependencies trees of the plugins, copying the .sos into place, along with the dependent dylibs and finally mangling the mach'o headers of both.

I also created an empty file /path/to/pyinstaller/PyInstaller/hooks/hook-gst.py to stop pyinstaller to complain about the missing parent hook. And, anyway, the hook code could go directly to hook-gst.py.

runtime hook file (rthook.py)

Finally I added a runtime hook file, referenced on the Analysis object, that sets up the environment variables that help gstreamer to locate the plugins. This code gets executed on the bundled executable before player.py (following the way Kivy sets up pyinstaller and thanks for their precious hint):

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys

if hasattr(sys, '_MEIPASS'):
    # PyInstaller >= 1.6
    root = sys._MEIPASS
elif '_MEIPASS2' in environ:
    # PyInstaller < 1.6 (tested on 1.5 only)
    root = os.environ['_MEIPASS2']
else:
    root = os.path.dirname(sys.argv[0])

os.chdir(root)

os.environ['GST_REGISTRY_FORK'] = 'no'
os.environ['GST_PLUGIN_PATH'] = os.path.join(root, 'gst-plugins')

It seems that disabling GST_REGISTRY_FORK is the only way to have a working outcome. Leaving the default setting (active) leads to a segmentation fault as soon as the first plugin gets scanned.

pyinstaller invocation

pyinstaller can be invoked with:

$ /path/to/pyinstaller/pyinstaller.py player.spec

Cool! If you haven't done so, I'd still suggest opening a hooks issue with `PyInstaller` submitting this as a patch, and/or starting a discussion on the mailing list. At minimum, you'll probably get someone who has experience with PyInstaller hooks to look over what you've done and validate that you did everything right. — abarnert, May 20 '13 at 17:49

pygst + pyinstaller under OSX

1 Answers1

pyinstaller spec file (player.spec):

_gst.so hook (hook-gst._gst.py):

runtime hook file (rthook.py)

pyinstaller invocation