How to use getattr or getattribute to correctly raise ImportError while calling some methods

Question

Hello their thanks in advance for helping me,

Please see below code:

import types

_MSG = ("Failed importing {name}. Please install {name}."
        " Using pip install {name}")

class Empty(): # pylint: disable=too-few-public-methods
    """Empty class for beam API."""
    def __call__(self, *args, **kwargs):
        return None

class DummyBeam(types.ModuleType): # pylint: disable=too-few-public-methods

    DoFn = Empty
    Pipeline = Empty

    def __init__(self, name="apache_beam"):
        super(DummyBeam, self).__init__(name)

    def __getattribute__(self, _):
        if getattr(DummyBeam, _, None) is Empty:

            err_msg = _MSG.format(name=self.__name__)
            raise ImportError(err_msg)

What I want to check if apache_beam was not installed it will successfully load all beam classes like DoFn and pipeline but calling some function raises error please see below code to see above code in use.

try:
  import apache_beam as beam
except ImportError:
  beam = DummyBeam()

class SomeFn(beam.DoFn):
  pass

class SomeOtherFn(beam.Pipeline):
  pass

SomeFn()

In above code for now accessing beam.DoFn raises error but what I want it to not raise error when accessing beam.DoFn although it raises error when calling SomeFn(). Also tried to replace getattribute with getattr and it not gives me results as i expected it wont raise error when calling SomeFn() although it runs fine for all codes.

Thanks for looking into this.

Answer 1

As shown in the traceback (that you should have posted FWIW), your error is not in calling SomeFn() but in accessing beam.DoFn in the SomeFn class definition. And the reason is quite obvious: you very explicitely instructed Python to do so by plain overriding Beam.__getattribute__.

Note that object.__getattribute__ is the official default implementation of attribute lookup (it's invoked each time Python sees either obj.name or getattr(obj, "name"), and that it's a method that is better left alone unless you fully understand the implications of overriding it AND have no better solution.

In this case the very obvious solution is to instead implempent __getattr__, which is only invoked by __getattribute__ as a last resort if the attribute couldn't be resolved in any other way. You say that:

Also replace getattribute with getattr was not working

but I just tried it on your code snippet and it (of course) yield the result I expected. Whether this is what you expected is another question, but since you neither posted this version of your code nor cared to explain how it "was not working", you can't expect any answer on this point (hint: "is not working" is a totally useless description of an issue).

As a last note:

it will successfully load all beam methods like DoFn and pipeline ... In above code for now calling beam.DoFn

It seems you're a bit confused about terminology. DoFn and Pipeline are classes, not methods, and (as already mentionned) your error is raised when accessing beam.DoFn, not when calling it.

EDIT:

by was not working I meant it not gives me error either when i am trying to access beam.DoFn or SomeFn() when use getattr instead getattribute (...) what i want is to raise error when calling someFn no accessing beam.DoFn

Ok, it looks that you don't quite get the execution order of a method call expression. When you do

obj.method()

this is actually a shortcut for

method = obj.__getattribute__("method")
method.__call__()

So overriding __getattribute__ isn't the proper solution (cf above), and defining __getattr__ is useless here - your DummyBeam class HAS DoFn and Pipeline attributes so __getattr__ will just not be invoked for those names.

Now the reason you don't get any exception when calling beam.DoFn or beam.Pipeline is that those name are bound to your Empty class, not instances of that class, so you actually never invoke Empty.__call__. Rhe __call__ method defined in a class is only used when an instance of that class is called, not when you instanciate the class (in which case it's the metaclass's __call__ method which is invoked):

>>> class MyCallable:    
...     def __init__(self):
...         print("in MyCallable.__init__")
...     def __call__(self):
...         print("in MyCallable.__call__")
... 
>>>  
... c = MyCallable()
in MyCallable.__init__
>>> c()
in MyCallable.__call__
>>> 

So if what you want is to raise your exception when someone tries to instanciate DoFn or ̀Pipelineyou either have to make them instances ofEmptyor keep them as they are and renameEmpty.calltoEmpty.newwhich is the first method called bytype.call(type` being the default metaclass for all classes).