RMOTR: Python Magic Methods and __getattr__

During our second week of our Advanced Python Programming course, we talk about the importance of good, clean and intuitive interfaces and how amazing the Python Object Oriented model is (a topic for another post). While we’re explaining this, we introduce a few useful magic methods like __len__, __str__, __repr__, etc. Among these, we also show both __getattr__ and __getattribute__ and seems like it’s always interesting for our students, so we thought we should share it with the world.

A primer on Magic Methods

Magic Methods are a broad and general term that refers to “special” methods in a Python class. There’s no single definition for all of them, as their use is diverse. For example, a few common and well known magic methods include:

• __init__ that serves as the object initializer (sometimes incorrectly referred to as constructor)
• __str__ that provides a “string representation” of your object
• __add__ that allows you to “overload” the + operator.

What all these methods have in common? Well, obviously they all start and end with double underscores (__). But aside from that, what makes them “magic methods” is that they’re invoked somehow “specially”. We don’t manually invoke these methods; Python is the one doing it. For example, we don’t do obj.__str__(), we do str(obj).

There are MANY magic methods, but as I said at the beginning, I want to focus on __getattr__ and __getattribute__.

__getattr__

Let’s start with __getattr__. This method will allow you to “catch” references to attributes that don’t exist in your object. Let’s see a simple example to understand how it works:

class Dummy(object):
    pass

d = Dummy()
d.does_not_exist  # Fails with AttributeError

In this example, the attribute access fails (with an AttributeError) because the attribute does_not_exist doesn’t exist.

But using the __getattr__ magic method, we can intercept that inexistent attribute lookup and do something so it doesn’t fail:

class Dummy(object):
    def __getattr__(self, attr):
        return attr.upper()

d = Dummy()
d.does_not_exist # 'DOES_NOT_EXIST'
d.what_about_this_one  # 'WHAT_ABOUT_THIS_ONE'

But if the attribute does exist, __getattr__ won’t be invoked:

class Dummy(object):
    def __getattr__(self, attr):
        return attr.upper()

d = Dummy()
d.value = "Python"
print(d.value)  # "Python"

__getattribute__

__getattribute__ is similar to __getattr__, with the important difference that __getattribute__ will intercept EVERY attribute lookup, doesn’t matter if the attribute exists or not. Let me show you a simple example:

class Dummy(object):
    def __getattribute__(self, attr):
        return 'YOU SEE ME?'

d = Dummy()
d.value = "Python"
print(d.value)  # "YOU SEE ME?"

In that example, the d object already has an attribute value. But when we try to access it, we don’t get the original expected value (“Python”); we’re just getting whatever __getattribute__ returned. It means that we’ve virtually lost the value attribute; it has become “unreachable”.

If you ever need to use __getattribute__ to simulate something similar to __getattr__ you’ll have to do some more advanced Python handling:

class Dummy(object):
    def __getattribute__(self, attr):
        __dict__ = super(Dummy, self).__getattribute__('__dict__')
        if attr in __dict__:
            return super(Dummy, self).__getattribute__(attr)
        return attr.upper()

d = Dummy()
d.value = "Python"
print(d.value)  # "Python"
print(d.does_not_exist)  # "DOES_NOT_EXIST"

A more realistic example

It’s not common to just randomly catch every attribute lookup. We generally use __getattr__ with a clear objective in mind: whenever we want to provide some dynamic access to our objects. A good example could be extending the base Python tuple to add it some Scala flavor to it (I really like Scala 🙌). In Scala, tuples are created really similarly to Python:

val a_tuple = ("z", 3, “Python”, -1)

But they’re accessed in a different way:

println(a_tuple._1) // “z”
println(a_tuple._3) // “Python”

Each element in the tuple is accessed as an attribute, with the first element being the attribute _1, the second _2, and so on.

We can easily extend our common Python tuple to match this behavior, the code is really simple:

http://ift.tt/2w7RHq7

You didn’t know you can extend builtin Python collections 😱? Check our Advanced Python Programming Course!

In that example, you can see how we’re catching missing attributes with __getattr__, but if that attribute is not in the form of _n (where n is an integer), we just raise the AttributeError manually.

Conclusion

Magic Methods are a great mechanism to extend the basic features of Python classes and objects and provide more intuitive interfaces. You can provide dynamic attribute lookups with __getattr__ for those missing attributes that you want to intercept. But be careful with __getattribute__, because it might be tricky to implement correctly without losing attributes in the Python void.

Special thanks to Fabrício Calado (@fabricalado) for proofreading and suggestions 🙇🏻.

---

Python Magic Methods and __getattr__ was originally published in rmotr.com on Medium, where people are continuing the conversation by highlighting and responding to this story.