How does eclipse's pydev do code completion?

Question

Does anyone know how pydev determines what to use for code completion? I'm trying to define a set of classes specifically to enable code completion. I've tried using __new__ to set __dict__ and also __slots__, but neither seems to get listed in pydev autocomplete.

I've got a set of enums I want to list in autocomplete, but I'd like to set them in a generator, not hardcode them all for each class.

So rather than

class TypeA(object):
    ValOk = 1
    ValSomethingSpecificToThisClassWentWrong = 4
    def __call__(self):
        return 42

I'd like do something like

def TYPE_GEN(name, val, enums={}):
    def call(self):
        return val
    dct = {}
    dct["__call__"] = call
    dct['__slots__'] = enums.keys()
    for k, v in enums.items():
        dct[k] = v
    return type(name, (), dct)

TypeA = TYPE_GEN("TypeA",42,{"ValOk":1,"ValSomethingSpecificToThisClassWentWrong":4})

What can I do to help the processing out?

edit: The comments seem to be about questioning what I am doing. Again, a big part of what I'm after is code completion. I'm using python binding to a protocol to talk to various microcontrollers. Each parameter I can change (there are hundreds) has a name conceptually, but over the protocol I need to use its ID, which is effectively random. Many of the parameters accept values that are conceptually named, but are again represented by integers. Thus the enum.

I'm trying to autogenerate a python module for the library, so the group can specify what they want to change using the names instead of the error prone numbers. The __call__ property will return the id of the parameter, the enums are the allowable values for the parameter.

Yes, I can generate the verbose version of each class. One line for each type seemed clearer to me, since the point is autocomplete, not viewing these classes.

You should not need to generate classes on the fly for this. Also, generating classes on the fly is probably not going to be compatible with autocompletion. — user2357112, Aug 12 '13 at 03:10
May I ask why you don't want to hard code the names? What benefit are you hoping to get by generating the enum values from a `dict` of strings? — jpmc26, Aug 12 '13 at 03:19
I don't know how smart pydev is, but you could try defining your enums via a metaclass. — martineau, Aug 12 '13 at 03:36
It's not any less hardcoded if you put everything in a dict literal and pass it to a constructor function than if you use regular class statements. You only lose clarity. — user2357112, Aug 12 '13 at 03:57
What are you trying to achieve by having instances of these classes be callable? What the heck would an instance of one of these classes be good for? — user2357112, Aug 12 '13 at 03:58
As a side note, it's a bad idea to have a mutable container as a default param (`enums={}`). Changes to it from inside the function will persist across calls. Much better is to have a default value of `None` and check for this inside the function. — detly, Aug 12 '13 at 23:05
@detly Generally it's simply a bad idea to change a mutable container which is an argument (unless the function's purpose is to mutate the container, but then it's generally obvious and doesn't have a default). Certainly here it would be a bug to mutate `enums` inside `TYPE_GEN` whether it's the default empty dictionary or one passed in from a caller. So it wouldn't actually prevent any bugs to use a `None` default here, and if it made you think it *was* safe to mutate `enums` then it would cause them. I no longer apply the `None` default pattern blindly, and have found my code simpler for it. — Ben, Aug 13 '13 at 11:28
@Ben - It depends. It might be sensible for something to mutate a `dict` passed in by the caller (a cache, a memoisation store, a namespace for an embedded script, etc). But I'd consider a mutable default argument *worse* than this simply because it's almost always unexpected behaviour. Obviously it's loose advice that can be disregarded by the experienced, but might well be useful to the OP. — detly, Aug 13 '13 at 11:41
@BrettStottlemyer If "viewing" is not the point, then it shouldn't matter what it looks like. Just go ahead and generate the verbose version. — Zoran Pavlovic, Nov 13 '14 at 08:19

score 1 · Answer 1 · answered Aug 15 '13 at 17:24

Ok, as pointed, your code is too dynamic for this... PyDev will only analyze your own code statically (i.e.: code that lives inside your project).

Still, there are some alternatives there:

Option 1:

You can force PyDev to analyze code that's in your library (i.e.: in site-packages) dynamically, in which case it could get that information dynamically through a shell.

To do that, you'd have to create a module in site-packages and in your interpreter configuration you'd need to add it to the 'forced builtins'. See: http://pydev.org/manual_101_interpreter.html for details on that.

Option 2:

Another option would be putting it into your predefined completions (but in this case it also needs to be in the interpreter configuration, not in your code -- and you'd have to make the completions explicit there anyways). See the link above for how to do this too.

Option 3:

Generate the actual code. I believe that Cog (http://nedbatchelder.com/code/cog/) is the best alternative for this as you can write python code to output the contents of the file and you can later change the code/rerun cog to update what's needed (if you want proper completions without having to put your code as it was a library in PyDev, I believe that'd be the best alternative -- and you'd be able to grasp better what you have as your structure would be explicit there).

Note that cog also works if you're in other languages such as Java/C++, etc. So, it's something I'd recommend adding to your tool set regardless of this particular issue.

score 0 · Answer 2 · answered Aug 12 '13 at 20:20

The simpler your code, the more likely completion is to work. Would it be reasonable to have this as a separate tool that generates Python code files containing the class definitions like you have above? This would essentially be the best of both worlds. You could even put the name/value pairs in a JSON or INI file or what have you, eliminating the clutter of the methods call among the name/value pairs. The only downside is needing to run the tool to regenerate the code files when the codes change, but at least that's an automated, simple process.

Personally, I would just go with making things more verbose and writing out the classes manually, but that's just my opinion.

On a side note, I don't see much benefit in making the classes callable vs. just having an id class variable. Both require knowing what to type: TypeA() vs TypeA.id. If you want to prevent instantiation, I think throwing an exception in __init__ would be a bit more clear about your intentions.

score 0 · Accepted Answer · answered Aug 12 '13 at 22:47

Fully general code completion for Python isn't actually possible in an "offline" editor (as opposed to in an interactive Python shell).

The reason is that Python is too dynamic; basically anything can change at any time. If I type TypeA.Val and ask for completions, the system had to know what object TypeA is bound to, what its class is, and what the attributes of both are. All 3 of those facts can change (and do; TypeA starts undefined and is only bound to an object at some specific point during program execution).

So the system would have to know st what point in the program run do you want the completions from? And even if there were some unambiguous way of specifying that, there's no general way to know what the state of everything in the program is like at that point without actually running it to that point, which you probably don't want your editor to do!

So what pydev does instead is guess, when it's pretty obvious. If you have a class block in a module foo defining class Bar, then it's a safe bet that the name Bar imported from foo is going to refer to that class. And so you know something about what names are accessible under Bar., or on an object created by obj = Bar(). Sure, the program could be rebinding foo.Bar (or altering its set of attributes) at runtime, or could be run in an environment where import foo is hitting some other file. But that sort of thing happens rarely, and the completions are useful in the common case.

What that means though is that you basically lose completions whenever you use "too much" of Python's dynamic language flexibility. Defining a class by calling a function is one of those cases. It's not ready to guess that TypeA has names ValOk and ValSomethingSpecificToThisClassWentWrong; after all, there's presumably lots of other objects that result from calls to TYPE_GEN, but they all have different names.

So if your main goal is to have completions, I think you'll have to make it easy for pydev and write these classes out in full. Of course, you could use similar code to generate the python files (textually) if you wanted. It looks though like there's actually more "syntactic overhead" of defining these with dictionaries than as a class, though; you're writing "a": b, per item rather than a = b. Unless you can generate these more systematically or parse existing definition files or something, I think I'd find the static class definition easier to read and write than the dictionary driving TYPE_GEN.

How does eclipse's pydev do code completion?

3 Answers3