2

I wrote a small script. It's designed to search the python directory for all available modules (whether they are installed or not), then it is supposed to check what modules are currently loaded, then it offers an option to dynamically load a module of your choice. The latter using __import__() because I am passing a string to it - (this is where I am having a problem - but I'll get back to it shortly)...then it gives the option to "browse" the module for all its classes, functions, etc. (using dir([module name]) ...).

The problem:

When the module is loaded dynamically - it is embedded in a try/except statement - if it succeeds it reports that the "module is loaded" and if it fails it reports...duh..."Failed to load..."

If you type the name of a module, for example a module named "uu", it says "loaded". So I know it is loading - however, when I go back and call the function that checks all of the LOADED modules - it is blank (using sys.modules)

I am thinking that python is loading the module into a temporary place which is not sys.modules because when I exit out of the script and check sys.modules it is not there.

chb
  • 1,727
  • 7
  • 25
  • 47
Justin Carroll
  • 1,362
  • 1
  • 13
  • 37
  • paste the script here not the paste link – Anurag Uniyal Dec 28 '09 at 11:50
  • This script is too long to put directly into the question. – Georg Schölly Dec 28 '09 at 11:55
  • I cant paste the entire script - gs is right, its too long. But, it is not a long script I promise :P. I wouldn't know which parts are important enough to paste to illustrate the problem... but I can try to summarize: Essentially it reads like this: var1 = raw_input("Whats the module name you want to load?: ") __import__(var1) then I check it with sys.modules WHILE THE SCRIPT IS RUNNING, - which prints a LONG list of all the modules that are loaded, but the var1 (name of the module) is not in that list - and yes, I KNOW that it IS loading and is spelled correctly :P. – Justin Carroll Dec 28 '09 at 12:19
  • The script works for me. Maybe the `list_all_mods` module does wired things to `sys.modules` (I commented out `list_all_mods` to test the script)? – sth Dec 28 '09 at 12:25
  • sth - are you telling the script to LOAD a module that you know is not already loaded? For instance, sys (a module, duh :P) is loaded when the script starts up, but the function that I built will also 'reload' sys if you tell it to - then if you check using sys.modules it'll show up there. I found that a module named "uu" comes standard with the python install, and is not normally imported on startup (at least it is not on my install or my script) - try loading 'uu' and then check to see if it is in sys.modules (within the script using the option "2"). – Justin Carroll Dec 28 '09 at 12:29
  • After startup, option "2" lists 34 modules already loaded (things like `sys` and `site`, ...). Then I load `uu` with option "3". Listing the loaded modules again with "2" now shows 36 modules, `uu` and `binascii` (required by `uu`) are the new ones. – sth Dec 28 '09 at 12:36
  • Hmm, I went back and tested this in IDLE (I was using pyScripter and the interpreter within pyScripter) and it works like a charm... I wonder why pyScripter is reporting a totally different list... 8(. – Justin Carroll Dec 28 '09 at 12:37
  • sth, do you know why one IDE would report a different list using sys.modules as another IDE? But thanks for checking :P spent 4 days working on this "problem" and had trouble finding someone who would check my code! – Justin Carroll Dec 28 '09 at 12:39
  • @Nascent_Notes: No, no idea what these IDEs do to `sys.modules`. You should post a new question about these strange differences in the IDEs, maybe somebody else knows more about that. – sth Dec 28 '09 at 13:21
  • @GeorgSchölly And now the script is gone because the external service that hosted it went belly-up. – chb Dec 13 '17 at 19:29

1 Answers1

1

Nascent_Notes, nice script! I tried loading uu (command 3) and printing the list of loaded modules (command 2) and they both seem to work fine.

However, if I try to "browse the module" (command 4), I get the following error:

HlpWiz>>> 4
What module do you want to look more into?:  uu

*An error occurred - probably because the module isn't loaded or is misspelled*

Try running

#!/usr/bin/env python
import sys
__import__('uu')
print(sys.modules['uu'])
print(dir(uu))

You should get NameError: name 'uu' is not defined.

So it appears that although __import__ successfully imports the uu module, it does not add uu to the global namespace -- the module uu can not be accessed by the variable name uu. It can be accessed through sys.modules however:

Therefore, change

    var_mod = input("What module do you want to look more into?:  ")
    print "\n attempting to browse... please wait!"
    time.sleep(2)
    browse_mod(zlib = var_mod)

to

    var_mod = raw_input("What module do you want to look more into?:  ")
    print "\n attempting to browse... please wait!"
    time.sleep(2)
    browse_mod(zlib = sys.modules[var_mod])

Not only is using raw_input much safer than input (the user will not be able to execute unexpected/malicious commands), but also raw_input does what you want here.

On a minor note, you could also change

i = 1
for line in sample:
    print i, line
    i = i + 1

to the more pythonic

for i,line in enumerate(sample):
    print i+1, line

Edit:

sys.modules is a dict (short for dictionary). Dicts are like telephone books -- you give it a name (better known as a "key") and it returns a phone number (or more generally, a "value").

In the case of sys.modules, the keys are module names (strings). The values are the module objects themselves.

You access the values in the dict using bracket notation. So uu is just a string, but sys.modules['uu'] is the module uu.

You can read the full story on dicts here: http://docs.python.org/tutorial/datastructures.html#dictionaries

unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • Thanks!!! That worked like a charm!! You are a god-send! I was trying variants of the i+1, I tried i++ and that didn't work - so I fell back on what I knew would work, didn't think to just put the expression where you suggested - thanks! Also, I knew that "input" was dangerous to use, as per pydocs, but it was the only way I could get it to not set the value assigned to the variable as a string - but you solved that by doing sys.modules[var_mod] - what do to the "[]" do? how is it passing the string? or is it just 'appending' the module to sys.modules? - confused... :P Thanks btw! – Justin Carroll Dec 28 '09 at 13:06
  • @Nascent_Notes, I've added a bit of explanation regarding sys.modules and dicts. Hope it helps. – unutbu Dec 28 '09 at 13:39
  • k duh - Im stupid lol sorry - yeah I know dicts, I didnt realize that sys.modules was a dict, so I thought that you were doing something 'special' - thanks for explaining that! Taught myself the value of type() :P. Hopefully that will help in the future... – Justin Carroll Dec 28 '09 at 14:01