ImportError: No module named bs4 when called from launchd

Question

I am running the following python imports in a script that works fine from the command line in terminal and from the ipython notebook.

#!/usr/bin/python
import os
import re
import urllib
import urllib2 as ul
import sys
from bs4 import BeautifulSoup as bs

When it's called from a .plist file via Mac launchd I get the following error:

11/24/15 1:17:05 PM com.jerry.sat_images[668]   Traceback (most recent call last):
11/24/15 1:17:05 PM com.jerry.sat_images[668]     File "/Users/jerrykallam/python_practice/sat_image.py", line 6, in <module>
11/24/15 1:17:05 PM com.jerry.sat_images[668]       import bs4
11/24/15 1:17:05 PM com.jerry.sat_images[668]   ImportError: No module named bs4
11/24/15 1:17:05 PM com.apple.launchd.peruser.501[165]  (com.jerry.sat_images[668]) Exited with exit code: 1

From the command line and ipython bs4 imports and the script works fine. This is the .plist code that seems to work correctly. Not sure why the script fails to import bs4 only when called by launchd.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.jerry.satimages</string>
        <key>ProgramArguments</key>
        <array>
           <string>python</string>
            <string>/Users/jerrykallam/python_practice/sat_image.py</string>
        </array>
        <key>StartInterval</key>
        <integer>360</integer>
        <key>RunAtLoad</key>
        <true/>
    </dict>
</plist>

Can you run `sudo pip freeze | grep beautifulsoup4` and see if the package is installed for the superuser? — wpercy, Nov 24 '15 at 23:37
I get the following:The directory '/Users/jerrykallam/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. beautifulsoup4==4.3.2 — cubflier, Nov 24 '15 at 23:40
When I run sudo -H pip freeze | grep beautifulsoup4, I get: beautifulsoup4==4.3.2 — cubflier, Nov 24 '15 at 23:45
Permissions are: drwx------ 4 jerrykallam staff 136 Oct 26 14:53 http — cubflier, Nov 24 '15 at 23:47
When I run 'echo $PYTHONPATH' nothing is returned so I guess there is no path set. — cubflier, Nov 25 '15 at 00:00

score 2 · Answer 1 · answered Mar 14 '16 at 22:40

I ran into this exact problem (OS X 10.11.3 El Cap). Adding PYTHONPATH to EnvironmentVariables did not work. What did work is calling the specific path for python.

So, in ProgramArguments instead of:

   <string>python</string>

... this:

   <string>/your/python/path</string>

score 1 · Answer 2 · answered Nov 24 '15 at 23:45

Look for differences in sys.path between the CLI and when it's run in from the plist. If the bs4 module isn't in the right place relative to the python path it's not going to find it.

 #!/usr/bin/env python

 import sys
 print sys.path

With the plist version you might have to:

fd=open('/var/tmp/fred.txt','w') 

ted = sys.path

fd.writelines(ted) 
fd.close()

instead of just "print sys.path".

Basically I'm guessing that the .plist file is getting run before your unix(ish) environment is fully set up.

score 1 · Accepted Answer · answered Nov 24 '15 at 23:49

This is likely because launchd runs its daemons as root, and bs4 is likely not installed in a location in the super user's PYTHONPATH. To fix this, you can add an EnvironmentVariables key, and set the value of your PYTHONPATH there. To figure out what your PYTHONPATH is, you can run echo $PYTHONPATH.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.jerry.satimages</string>
        <key>ProgramArguments</key>
        <array>
           <string>python</string>
            <string>/Users/jerrykallam/python_practice/sat_image.py</string>
        </array>
        <!-- ADDED THIS -->
        <key>EnvironmentVariables</key>
        <dict>
            <key>PYTHONPATH</key>
            <string>/your/python/path</string>
        </dict>
        <key>StartInterval</key>
        <integer>360</integer>
        <key>RunAtLoad</key>
        <true/>
    </dict>
</plist>

ImportError: No module named bs4 when called from launchd

3 Answers3