21

I've been using virtualenv + pip for python development. I'm not sure what happened, but suddenly whenever I try to run a command-line tool or import libraries, I get this error message:

Traceback (most recent call last):
  File "/Users/kyle/.virtualenvs/fj/bin/pip", line 4, in <module>
    import pkg_resources
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 698, in <module>
    the platform/python version defined at initialization are added.
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 701, in Environment
    search_path = sys.path
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 96, in get_supported_platform
    'Environment', 'WorkingSet', 'ResourceManager',
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 221, in get_build_platform
    if provDarwin:
  File "/Users/kyle/.virtualenvs/fj/lib/python2.6/distutils/__init__.py", line 14, in <module>
    exec open(os.path.join(distutils_path, '__init__.py')).read()
IOError: [Errno 2] No such file or directory: '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/__init__.py'

From what I can decipher, Python is trying to find distutils_path in the Mac OSX system version Python, not my virtualenv version like it should be.

Not sure why this suddenly started happening. Maybe a recent OSX update? Another possibility is that my hard drive was about to die, so Apple gave me a new one and ran Migration Assistant. Maybe something didn't transferred across correctly?

Kyle Fox
  • 3,133
  • 4
  • 23
  • 26
  • After looking at distutils/__init__.py, it's doing this: `distutils_path = os.path.join(os.path.dirname(ConfigParser.__file__), 'distutils')` I really have no idea how this works, but it appears `ConfigParser` is pointing to OSX system python, when it should be pointing to my virtualenv. – Kyle Fox Jun 28 '10 at 03:15
  • Don't know if this will help anyone but, Homebrew has a good install of python and virtualenv check this out. https://gist.github.com/pithyless/1208841 – earlonrails Aug 10 '13 at 17:36

5 Answers5

33

I encountered this distutils/__init__.py problem when transitioning to OS X 10.7 Lion (from OS X 10.5 Leopard) and using Migration Assistant. I've already installed Xcode 3.2.6 -- thus resolving the missing install_name_tool problem.

Migration Assistant brought over my previous virtualenvs, but since they were based on Leopard's Python 2.5, I figure I need to recreate each of them with the current system Python 2.7.

easy_install was already in the PATH -- probably because it was bundled with Lion's Python 2.7; it seems unlikely to be the result of Migration Assistant. I used easy_install to install virtualenv.

This problem, it seems to me, doesn't have anything to do with Xcode or lack thereof. It's a peculiar line in a file placed in the new virtual env by the virtualenv command:


  File "/path/to/my/virtualenv/lib/python2.7/distutils/__init__.py", line 16, in 
    exec(open(os.path.join(distutils_path, '__init__.py')).read())
IOError: [Errno 2] No such file or directory: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/__init__.py'

The issue is that, in the Python 2.7 install bundled with Lion, the library doesn't come with .py source files. That directory contains .pyc and .pyo files, but no .py files. virtualenv doesn't seem to expect that.

My workaround is to download Python 2.7 source:
http://python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2

and unpack distutils/__init__.py into the expected place:
sudo tar xvjf ~/Downloads/Python-2.7.2.tar.bz2 --strip-components=2 -C /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7 Python-2.7.2/Lib/distutils/__init__.py

That permits virtualenv to complete successfully, and the resulting Python interpreter seems to run.

Given that the Python 2.7 library bundled with Lion is installed without source, it might seem useful to change virtualenv to try for either distutils/__init__.py or distutils/__init__.pyc ?

Nat Goodspeed
  • 441
  • 5
  • 3
  • 17
    Am I crazy? Just creating the file worked for me, a la `sudo touch __init__.py` – npdoty Mar 09 '12 at 07:05
  • Hmm, thanks! I didn't try that: I assumed there was some point to virtualenv's wanting to read distutils/__init__.py, e.g. some particular symbol it wanted to see defined. Glad to know there's an even simpler solution than mine. – Nat Goodspeed Apr 02 '12 at 21:45
  • seems like someone should file a bug report with virtualenv...? – metamatt Jul 20 '12 at 18:48
  • Quick reading of the mentioned `__init__.py` shows only comments and metadata, that's why the `touch` [...] method works. Add a `__version__` module attribute if you're paranoid. – Jacob Oscarson Aug 06 '12 at 13:16
22
> cd /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/
> sudo touch __init__.py

Out-of-the-box python on Lion comes without the python source - just the compiled pyc/pyo files. However virtualenv goes looking for the distutils source file just to confirm where it is. Turns out all we need to do it touch the file it's looking for into existence.

Credits belong to "npdoty" and "Nat Goodspeed".

John Mee
  • 50,179
  • 34
  • 152
  • 186
  • 5
    could just be "sudo touch /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/__init__.py" – mikelikespie Jun 15 '12 at 02:22
  • Do NOT use this method! This can deal with the majority of cases, but there are situations where you will get in more trouble. The original file is not empty! It contains __version__ and __revision__ attributes and some scripts depend on those! Download the original source code as suggested by Nat. – LeZuse Nov 30 '12 at 18:40
  • @LeZuse Can you name the scripts please, so people can decide if it will or won't work for them. Thx. – John Mee Dec 02 '12 at 01:25
  • While running virtualenv --distribute venv (version 1.8.4) I got: `AttributeError: 'module' object has no attribute '__version__'` – LeZuse Dec 03 '12 at 13:25
7

Turns out the problem was that Migration Assistant, for whatever reason, didn't copy over tools like gcc -- I reinstalled Xcode and things work properly again.

Kyle Fox
  • 3,133
  • 4
  • 23
  • 26
  • Migration Assistant's job is to migrate *user accounts*, not the whole OS and associated software. – Eric O. Lebigot Jun 28 '10 at 04:11
  • so, just a question, you just installed xcode again, and it works fine?! this seems weird for me, anyways, I am getting the same problem after I upgrade the Mac OS to Lion. so I am downloading a new version of xcode and will try to install it, to see.! if by any chance you did also something else, please lemme know. cheers – Arthur Neves Jul 23 '11 at 20:16
  • This works: after using Migration Assistant to set up a Mac, I used Kenneth Reitz' GCC installer (the lightweight alternative to installing XCode) and this problem went away. https://github.com/kennethreitz/osx-gcc-installer – Joe Germuska Jun 04 '12 at 18:58
  • You need the command line utils of xcode. Found the answer here: http://stackoverflow.com/a/11193009/268125 – j7nn7k Aug 06 '12 at 13:32
1

While Migration Assistant doesn't handle things like XCode so well, it is nevertheless designed to transfer some types of applications. It works best with those that exist entirely in the /Applications/ folder or those applications that launch from the /Applications/ folder and do checks for associated files located elsewhere (e.g., /usr/bin/), installing them when they aren't detected on startup.

See http://support.apple.com/kb/HT4413.

Wes
  • 171
  • 2
0

I have used a similar approach of Nat Goodspeed.

But I've copied all *.py files.

Download the same version of your system python, 2.7.2 in my case:

$ python --version

Download it and unpack it. http://python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2

# -n copy only missing files, -r recursively
$ sudo cp -rn ~/Downloads/Python-2.7.2/Lib/* /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
lucmult
  • 51
  • 1
  • 2