1

We've been doing a fair amount of Python scripting, and now we have a directory with almost a hundred loosely related scripts. It's obviously time to organize this, but there's a problem. These scripts import freely from each other and although code reuse is generally a good thing it makes it quite complicated to organize them into directories.

There's a few things that you should know about our corporate environment:

  1. I don't have access to the users' environment. Editing the PYTHONPATH is out, unless it happens in the script itself.
  2. Users don't install things. Systems are expected to be already installed and working on, so setup.py is not a solution unless I can run it once for all users.

I'm quite willing to edit my import statements and do some minor refactoring, but the solutions I see currently require me to divide all the code strictly between "user runnable scripts" and "libraries", which isn't feasible, considering the amount of code.

Has anyone out there solved a similar problem? Are you happy with it?

--Buck

Another way to state the same question:

When looking at google code search, this kind of code is rampant (below). Is everyone happy with this? Is there a good alternative?

sys.path.insert(0, os.path.dirname(os.path.dirname(
    os.path.dirname(os.path.abspath(__file__))
)))
bukzor
  • 37,539
  • 11
  • 77
  • 111
  • It sounds like you have created a mess. Libraries should always be separate from executable scripts. This is the very principal concept of code re-use. You need to refactor, do it now before it's 1,000 scripts. – Ben DeMott Mar 12 '11 at 18:24
  • @Ben DeMott: how do you know what's library and what's regular code when you're writing it? If it was as simple as rewriting all the import statements, there'd be no question, but the "library" and "regular" code is intermingled to the point that I don't see a way to tell the difference. Is the rule to never import stuff from a script directly? – bukzor Mar 13 '11 at 05:02
  • Can you please give more details about how these scripts are used? My first question would've been if implementing a version control system would be useful to your situation so that users can checkout the scripts they need, but it's possible I'm interpreting your situation the wrong way. At the moment, how are the users accessing the files? –  Mar 12 '11 at 17:52
  • Scripts live on a shared disk at a well-known location "/tool/mycompany/myteam/myproject/bin", and are sometimes run directly as utilities, or used implicitly as part of the infrastructure. We use version control as a matter of course, but mucking with vcs is beyond the scope of normal work for our users. In total there are maybe 600 people running these things on a daily basis, spanning five teams with completely different environments and methodologies. I don't quite see how any of this is relevant, so I'm adding it here rather than editing the question. – bukzor Mar 12 '11 at 17:58
  • I forgot to say "thanks", so ... thanks! =D – bukzor Mar 12 '11 at 18:00
  • I keep thinking, but unfortunately I don't see a quick way out of this. I'm afraid to ask approx how many scripts are we talking about, you'll probably have to refactor in the end. Or give the users read-only access to the vcs system and let them checkout everything they want, even the entire codebase if needed so they don't run into dependencies problems. –  Mar 12 '11 at 18:34
  • @jakker: thanks for thinking of me :) It's in the OP: almost a hundred scripts. Read-only vcs is not really different from what we have now, since the shared disk is kept at the HEAD revision. – bukzor Mar 12 '11 at 22:36
  • Sorry, somehow I missed the 'almost a hundred' thing, but dude, get refactoring :) ! Be glad it's not 200 or more, if it's only about modifying import statements and minor stuff like that, it's doable, crazy boring but you can probably finish it in 1 day if you're determined. And then never mix libraries with regular code again! :) Good luck! –  Mar 12 '11 at 22:40
  • @jakker: how do you know what's library and what's regular code when you're writing it? If it was as simple as rewriting all the import statements, there'd be no question, but the "library" and "regular" code is intermingled to the point that I don't see a way to tell the difference. Is the rule to never import stuff from a script directly? – bukzor Mar 13 '11 at 00:52
  • Well, for instance when I code, I refactor several times and make it a rule to import as little of my own code as possible, so this forces me to plan and group everything that might resemble a 'library'. I wouldn't like to clean up hundreds of files either. I'm probably not being very helpful with my comments right now. –  Mar 13 '11 at 09:29

0 Answers0