2

I'm working on an Inno Setup installer for a Python application for Windows 7, and I have these requirements:

  1. The app shouldn't write anything to the installation directory
  2. It should be able to use .pyc files
  3. The app shouldn't require a specific Python version, so I can't just add a set of .pyc files to the installer

Is there a recommended way of handling this? Like give the user a way to (re)generate the .pyc files? Or is the shorter startup time benefit from the .pyc files usually not worth worrying about?

user1989333
  • 21
  • 2
  • What versions of Python are you supporting? You can tell it to put the `.pyc` files in a separate location—like, say, your per-user Application Data directory instead of the installation directory, but this is easier with newer Python versions than older ones. – abarnert Jan 18 '13 at 21:24
  • As a side note, this is kind of a strange design. Usually when you need a Windows installer for your app, you don't rely on the user having installed a compatible Python in advance and try to work with that, you just `py2exe` or `cx_freeze` the whole thing up. Is there a reason you're doing things this way? How are you handling discovering the user's Python (or choosing from among multiple installations), organizing your dependencies, etc.? – abarnert Jan 18 '13 at 21:27
  • Currently Python 2.7. Putting the files in another directory that's writeable would solve one problem. But would it recreate the security problem UAC is supposed to solve? – user1989333 Jan 18 '13 at 22:23
  • No, the whole point of UAC is that it lets you write to safe per-user-and-per-process locations; it's only if you try to write outside such locations (like into your install directory) that there's a security problem. – abarnert Jan 18 '13 at 22:28
  • It's not my design, and I think it just sort of evolved this way. Now it needs to evolve some more. Currently we check the version of whatever Python runs when you type in "python". And we test to make sure certain required libraries are installed. There aren't a lot of installations, so this works good enough. – user1989333 Jan 18 '13 at 22:34

3 Answers3

0

PYC files aren't guaranteed to be compatible for different python versions. If you don't know that all your customers are running the same python versions, you really don't want to distribute pyc's directly. So, you have to choose between distributing PYCs and supporting multiple python versions.

You could create build process that compiles all your files using py_compile and zips them up into a version-specific package. You can do this with setuptools.; however it will be awkward to do because you'll have to run py_compile in every version you need to support.

If you are basically distributing a closed application and don't want people to have trivial access to your source code, then py2exe is probably a simpler alternative. If your python is supposed to be integrated into the user's python install, then it's probably simpler to just create a zip of your .py files and add a one-line .py stub that imports the zipped package(s) using zipfile

if it makes you feel better, PYC doesn't provide much extra security and it doesn't really boost perf much either :)

Community
  • 1
  • 1
theodox
  • 12,028
  • 3
  • 23
  • 36
  • The OP already acknowledged almost all of this in the question. He said he can't just distribute the .pyc files, because he wants to support multiple versions, so you don't need to explain that. He also clearly said he wants .pyc files for faster startup, not for security. – abarnert Jan 18 '13 at 22:29
  • I thought about the option of distributing different .pyc versions, but that seemed too complicated for our purposes. And we couldn't include versions that don't exist yet, so that would be even more complicated. – user1989333 Jan 18 '13 at 23:35
  • The short story is: pyc isn't worth the hassle except in very limited circumstances... for example, I distribute a python toolset to a bunch of 3d artists using Maya as a zipped package of pycs - but only because I know that are all on exactly the same python as I am. If I have to support multiple versions I'll revert to shipping PY. The only real benefit is for the first time startup on a given machine (in my case, its magnified a bit by the zip as well). – theodox Jan 19 '13 at 00:38
  • I'm leaning toward generating the .pyc files with the installer. But we probably wouldn't lose much by just ignoring them. And the user could still generate them if they wanted to. – user1989333 Jan 19 '13 at 04:59
  • The pycs will be generated appropriately the first time the script get executed - and after that you'll be in the same place so you're only saving time on the first run. – theodox Jan 19 '13 at 05:37
  • @theodox - That's how we currently get the .pyc files. But now we want to make it work for read-only installation directories, and the files won't get generated during normal use. But we should be able to generate them at install time with elevated privileges. – user1989333 Jan 20 '13 at 19:25
  • you could do it on the installing machine with a onetime script that sets the privileges and compiles the pycs. it's the privileges which will be the pain in any case... – theodox Jan 20 '13 at 19:38
0

If you haven't read PEP 3147, that will probably answer your questions.

I don't mean the solution described in that PEP and implemented as of Python 3.2. That's great if your "multiple Python versions" just means "3.2, 3.3, and probably future 3.x". Or even if it means "2.6+ and 3.1+, but I only really care about 3.2 and 3.3, so if I don't get the pyc speedups for other ones that's OK".

But when I asked your supported versions, you said, "2.7", which means you can't rely on PEP 3147 to solve your problems.

Fortunately, the PEP is full of discussion of earlier attempts to solve the problem, and the pitfalls of each, and there should be more than enough there to figure out what the options are and how to implement them.

The one problem is that the PEP is very linux-centric—mainly because it's primarily linux distros that tried to solve the problem in the past. (Apple also did so, but their solution was (a) pretty much working, and (b) tightly coupled with the whole Mac-specific "framework" thing, so they were mostly ignored…)

So, it largely leaves open the question of "Where should I put the .pyc files on Windows?"

The best choice is probably an app-specific directory under the user's local application data directory. See Known Folders if you can require Vista or later, CSIDL if you can't. Either way, you're looking for the FOLDERID_LocalAppData or CSIDL_LOCAL_APPDATA, which is:

The file system directory that serves as a data repository for local (nonroaming) applications. A typical path is C:\Documents and Settings\username\Local Settings\Application Data.

The point is that it's a place for applications to store data that's separate for each user (and inside that user's profile directory), and also for each machine the user's roaming profile might end up on, which means you can safely put stuff there and know that the user has the permissions to write there without UAC getting involved, and also know (as well as you ever can) that no other user or machine will interfere with what's there.

Within that directory, you create a directory for your program, and put whatever you want there, and as long as you picked a unique name (e.g., My Unique App Name or My Company Name\My App Name or a UUID), you're safe from accidental collision with other programs. (There used to be specific guidelines on this in MSDN, but I can no longer find them.)

So, how do you get to that directory?

The easiest way is to just use the env variable %LOCALAPPDATA%. If you need to deal with older Windows, you can use %USERPROFILE% and tack \Local Settings\Application Data onto the end, which is guaranteed to either be the same, or end up in the same place via junctions.

You can also use pywin32 or ctypes to access the native Windows APIs (since there are at least 3 different APIs for this and at least two ways to access those APIs, I don't want to give all possible ways to write this… but a quick google or SO search for "pywin32 SHGetFolderPath" or "ctypes SHGetKnownFolderPath" or whatever should give you what you need).

Or, there are multiple third-party modules to handle this. The first one both Google and PyPI turned up was winshell.

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • Thanks. I read 3147, and it seems to mainly be concerned with supporting multiple versions of python simultaneously. But we really only need one at a time. We just don't know exactly which one it'll be. It may even be a future version that doesn't exist yet, at the time the installer is created. So the .pyc files need to be generated after installation. But they can't be created in a folder that's read-only, and one of my requirements is to keep the installation folder read-only. So my job is to find out if there's a recommended way to deal with these seemingly conflicting requirements. – user1989333 Jan 19 '13 at 00:32
  • @user1989333: Well, as of 3.2, the recommended way is to put them in the appropriate cache directory, which Python defines for you. Since you need to work with 2.7, you have to invent some "cache directory" of your own, which is where `%LOCALAPPDATA%` comes in. Unless the user's done something really stupid (in which case you can fall back disable .pyc writing), you can always create a directory there, and write whatever you want to that directory, so there's your Windows solution. (If you care about other platforms, you'd need a bit more code… but you don't.) – abarnert Jan 19 '13 at 00:49
  • If the .py files are in a read-only directory under "c:\Program Files", and the user tries running the .py files from that read-only directory, is there a way to tell python 2.7 to create and use .pyc files in %LOCALAPPDATA%? – user1989333 Jan 19 '13 at 01:04
  • @user1989333: I think you can hook the compile method, or hook the importer to substitute your own method… but much simpler, just disable it (e.g., with `-B` or `PYTHONDONTWRITEBYTECODE=1`) and write your own code that uses [`py_compile`](http://docs.python.org/2/library/py_compile.html). If that sounds complicated, it's not—look at the source to [compileall.py](http://hg.python.org/cpython/file/2.7/Lib/compileall.py). – abarnert Jan 19 '13 at 01:44
0

Re-reading the original question, there's a much simpler answer that probably fits your requirements.

I don't know much about Inno, but most installers give you a way to run an arbitrary command as a post-copy step.

So, you can just use python -m compileall to create the .pyc files for you at install time—while you've still got elevated privileges, so there's no problem with UAC.

In fact, if you look at pywin32, and various other Python packages that come as installer packages, they do exactly this. This is an idiomatic thing to do for installing libraries into the user's Python installation, so I don't see why it wouldn't be considered reasonable for installing an executable that uses the user's Python installation.

Of course if the user later decides to uninstall Python 2.6 and install 2.7, your .pyc files will be hosed… but from your description, it sounds like your entire program will be hosed anyway, and the recommended solution for the user would probably be to uninstall and reinstall anyway, right?

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • Yes, Inno will let me do that. I was thinking it would require admin rights to get it to generate the .pyc files, but you're probably right that it already has the necessary elevated rights. And the fact that pywin32 does it this way, might be good enough to call it a "recommended" method. I was also thinking I'd need to add a function for regenerating after upgrading python. But a regular reinstall over the existing installation should work. I already proposed generating the files during installation, but this extra stuff should help. I think this is what I needed. Thanks! – user1989333 Jan 19 '13 at 03:24
  • @user1989333: Yeah, the problem with the "regenerate" script is that you'd (usually) have to run it with elevated privileges, and you'd need to communicate that to the user, and so on. That might be worth having available as a power-user feature, but you're right that just telling people to re-run the installer is probably the simplest answer for most users. – abarnert Jan 22 '13 at 19:56