2

I'm developing a Python utility module to help with file downloads, archives, etc. I have a project set up in a virtual environment along with my unit tests. When I want to use this module on the same computer (essentially as "Production"), I move the files to the mymodule directory in the ~/dev/modules/mymodule

I keep all 3rd-party modules under ~/dev/modules/contrib. This contrib path is on my PYTHONPATH, but mymodule is NOT because I've noticed that if mymodule is on my PYTHONPATH, my unit tests cannot distinguish between the "Development" version and the "Production" version. But now if I want to use this common utility module, I have to manually add it to the PYTHONPATH.

This works, but I'm sure there's a better, more automated way.

What is the best way to have a Development and Production module on the same computer? For instance, is there a way to set PYTHONPATH dynamically?

Matt Norris
  • 8,596
  • 14
  • 59
  • 90
  • Your question isn't very clear. Is the common utility module something you develop ? If it so then do you want to move it into the production area after the unit tests and use it from the production area for unit testing of other modules ? – sateesh Oct 12 '10 at 12:07
  • Yes, it is something I develop. I want to move it to Prod for use in other scripts and programs. – Matt Norris Oct 13 '10 at 11:54

3 Answers3

1

I'm guessing by virtual environment you mean the virtualenv package?

http://pypi.python.org/pypi/virtualenv

What I'd try (and apologies if I've not understood the question right) is:

  • Keep the source somewhere that isn't referenced by PYTHONPATH (e.g. ~/projects/myproject)
  • Write a simple setuptools or distutils script for installing it (see Python distutils - does anyone know how to use it?)
  • Use the virtualenv package to create a dev virtual environment with the --no-site-packages option - this way your "dev" version won't see any packages installed in the default python installation.
  • (Also make sure your PYTHONPATH doesn't have any of your source directories)

Then, for testing:

  • Activate dev virtual environment
  • Run install script, (usually something like python setup.py build install). Your package ends up in /path/to/dev_virtualenv/lib/python2.x/site-packages/
  • Test, break, fix, repeat

And, for production:

  • Make sure dev virtualenv isn't activated
  • Run install script
  • All good to go, the "dev" version is hidden away in a virtual environment that production can't see...
  • ...And there's no (direct) messing around with PYTHONPATH

That said, I write this with the confidence of someone who's not actually tried setting using virtualenv in anger and the hope I've vaguely understood your question... ;)

Community
  • 1
  • 1
FredL
  • 961
  • 7
  • 13
  • I do use virtualenv for developing, and the source dir is not referenced in my global PYTHONPATH. The problem, however, is that my global PYTHONPATH is maintained even in the activated virtualenv. So if I have the Prod code in my path, the tests look there first (its dir is more shallow). Is there a way to overwrite the global PYTHONPATH when activating virtualenv? Or perhaps I can just remove the Prod path at the start of my test? – Matt Norris Oct 13 '10 at 11:53
  • 1
    If you've only got the one directory on your PYTHONPATH then you could just add a line to the virtualenv activate script like "unsetenv PYTHONPATH" but if you've got several virtualenvs and/or entries on PYTHONPATH that could get a bit messy. Can you get your production version installed somewhere where you don't need to use PYTHONPATH to load it (thus avoiding the need to set PYTHONPATH in the first place), or are you doing that already..? (Again, apologies if I'm still not getting it!) – FredL Oct 13 '10 at 12:29
1

You can add/modify python paths at sys.path, just make sure that the first path is the current directory ".", because some third-party modules rely on importing from the directory of the current module.

More information on python paths: http://djangotricks.blogspot.com/2008/09/note-on-python-paths.html

Aidas Bendoraitis
  • 3,965
  • 1
  • 30
  • 45
  • Thank you for your help. I posted a follow-up question here http://stackoverflow.com/questions/3947464 because I'm having trouble getting nose to use your suggestion. – Matt Norris Oct 16 '10 at 02:45
0

You could set the PYTHONPATH as a global environment variable pointing to your Production code, and then in any shell in which you want to use the Development code, change the PYTHONPATH to point to that code.

(Is that too simplistic? Have I missed something?)

RichieHindle
  • 272,464
  • 47
  • 358
  • 399
  • Does "global environment variable" mean simply setting the PYTHONPATH for the system? Can you provide an example of how to set the path (rather than just add to it) in Python code? – Matt Norris Oct 12 '10 at 12:41
  • 1
    @Wraith: Re. "global environment variable": yes, exactly. Re. "set the path in Python code": the Python path is in `sys.path` as a plain old Python list - you can manipulate it however you like (for the current Python session). – RichieHindle Oct 12 '10 at 12:44