Lecture 2: OOP: everything about inheritance, magic methods + metaclasses
Hello everyone! Let's talk about Object-Oriented Programming (OOP), but we're not going to delve into theoretical concepts, as some familiar concepts are not fully realized in Python.
Instead, we'll study only what you need to know specifically in the context of the Python language.
Inheritance
Let's start with inheritance. Class inheritance is used to change the behavior of a specific class, as well as to extend its functionality.
For example, let's say we have a ready-made class for a pet.
class Pet:
def __init__(self, name=None):
self.name = nameLet's imagine that we need to model the process of populating a planet with pets. But we're not interested in populating the planet with just any pets; we specifically want to populate it with dogs without changing the original Pet class.
To inherit the Pet class, we declare the Dog class and specify the parent class Pet in parentheses. The new class, created through inheritance, inherits all the attributes and methods of the parent class. In this case, the Pet class is the parent class, also known as the base class or superclass. The Dog class, on the other hand, is called a child class or subclass.
To change the behavior of the Dog class, we will override the __init__ method and add a new attribute breed, where we will store the dog's breed. In the new __init__ method, we will also call the parent class's initializer using the super() function. Additionally, we will add a new method say that will simply print a string.
class Dog(Pet):
def __init__(self, name, breed=None):
super().__init__(name)
self.breed = breed
def say(self):
print(f'{self.name}: woof-woof!')
dog = Dog( 'Rex', 'Doberman ')
dog.say()
# Output: Rex: woof-woof!Multiple Inheritance
In Python, inheritance from multiple parent classes is allowed, which is also known as multiple inheritance. This technique is often used to implement mixin classes. Suppose we need to export data about our objects (dogs) in JSON format, either to store this data on a hard drive or to transfer it over a network. We can solve this problem using mixin classes and multiple inheritance.
Let's declare an ExportJSON class and implement a method there that exports data in JSON format. Then we'll create a new class called ExDog, which will inherit from the Dog class and the new mixin class ExportJSON.
import json
class ExportJSON:
def to_json(self):
return json.dumps({
"name": self.name,
"breed": self.breed
})
class ExDog (Dog, ExportJSON):
pass
dog = ExDog("Rex", "Doberman")
print(dog.to_json())
""" Output: {"name":"\u0428\u0430\u0440\u0438\u043a", "breed":"\u0414\u043e
\u0431\u0435\u0440\u043c\u0430\u043d"}
"""On one hand, this may seem convenient and flexible, but multiple inheritance and the use of a large number of mixins can reduce code readability. Therefore, it's important not to overuse this approach and avoid creating too many mixin classes.
Every class in Python is a descendant of the object class. We can easily verify this by using the issubclass function.
>>> issubclass(int, object)
True
>>> issubclass (Dog, object)
True
>>> issubclass (Dog, Pet)
True
>>> issubclass (Dog, int)
FalseWe can also use the `isinstance` function to check if a specific object is an instance of a certain class.
>>> isinstance(dog, Dog)
True
>>> isinstance(dog, Pet)
True
>>> isinstance(dog, object)
TruePython allows us to build fairly complex class hierarchies through inheritance. We've constructed a somewhat complex hierarchy: there's a class `ExDog`, which we created using multiple inheritance from the `Dog` class and the `ExportJSON` mixin class. In turn, the `Dog` class inherits from the `Pet` class, and all other classes inherit from the `object` class. If we try to create an instance of the `ExDog` class and access the `name` attribute, how does Python search for this attribute within the existing class hierarchy?
For this purpose, Python has something called the Method Resolution Order (MRO), which is a separate topic for study. However, all you need to know is the order in which Python searches for the necessary attribute or method. This order can be obtained using the `__mro__` attribute. It shows that if we try to access the `name` attribute, Python will first look in the `ExDog` class, then in the `Dog` class, and after that, it will check the `Pet` class, where it will find the `name` attribute. This list is also called the linearization of the class, meaning Python sequentially searches for any attributes and methods in this list. If it goes through the entire list and doesn't find the desired attribute or method, an `AttributeError` exception will be raised.
Try to follow the attribute search chain here. Python first searches entirely through the left branch, then through the second branch, and so on, in case a class inherits from multiple others.
# Method Resolution Order
>>> ExDog.__mro_
(<class '__main__.ExDog'>, <class '__main__. Dog'>, <class
'__main__.Pet'>, <class '__main__.ExportJSON'>, <class 'object'>) Class Composition
In addition to single and multiple inheritance, Python offers an alternative approach called class composition. Let's recall the previous example where we had a Pet class, and we derived the Dog class from it. To enable objects of the Dog class to export data, we introduced a mixin class called ExportJSON.
The final class, ExDog, used multiple inheritance and derived from both the Dog class and ExportJSON. If we suddenly needed to export data not only in JSON format but also in another format (for example, XML), we would need another mixin class, ExportXML:
class ExportJSON:
def to_json(self):
pass
class ExportXML:
def to_xml (self):
pass
class ExDog (Dog, ExportJSON, ExportXML):
pass
dog = ExDog('Fox', 'Mops')
dog.to_xml()
dog.to_json()Let's imagine that we need to add several more methods for exporting data. In such a case, we would constantly have to modify the code of our `ExDog` class, adding new mixin classes, which could overly complicate the code. To avoid this, class composition is used.
We'll create a new class, `PetExport`, which will not be used for creating instances but only for inheritance:
Let's also recall the classes we already had before.
Now let's recreate the ExDog class without using multiple inheritance.
class ExDog (Dog):
def __init__(self, name, breed=None, exporter=None):
super().__init__(name, breed)
self._exporter = exporter
def export(self):
return self._exporter.export(self)Now let's try creating an instance of the `ExDog` class. Suppose we want the object of this class to be able to export its data in XML format. For this, we need to pass the appropriate exporter.
By the way, note that when using composition, the required object is created at the moment the specific program is executed.
dog = ('Rex', 'Mutt', exporter=ExportXML())
dog.export()There’s just a little bit left. Let's try to implement the methods for export in the initial class hierarchy.
With JSON, it's simple – we use the json module and the dumps method.
class ExportJSON(PetExport):
def export(self, dog):
return json.dumps({
"name": dog.name,
"breed": dog.breed,
})Now let's implement the export method in the ExportXML
class ExportXML(PetExport):
def export(self, dog):
return f"""<?xml version="1.0" encoding="utf-8"?>
<dog>
<name>{dog.name}</name>
<breed>{dog.breed}</breed>
</dog>"""However, it is generally inconvenient to define the export method every time. Let's slightly modify the ExDog class and set a default export method.
We'll also add a check to see if the passed object is an instance of the PetExport class and whether it can perform data export at all. For this, we can use the isinstance function.
What should we do if the object passed to the function cannot perform the export? In this case, we'll raise an exception (specifically, ValueError) – this will indicate that the program cannot continue its work and will be stopped.
class ExDog(Dog):
def __init__(self, name, breed=None, exporter=None):
super().__init__(name, breed)
self._exporter = exporter or ExportJSON()
if not isinstance(self._exporter, PetExport):
raise ValueError('bad export instance value', exporter)
def export(self):
return self._exporter.export(self)Now, if we ever need to add a new export method to the code, we can easily do so. We just need to declare a new class, add it to the existing export hierarchy, and we won't need to change the ExDog class. We will be able to export to various formats easily and conveniently in the final program by simply substituting the desired exporter or creating one.
Magic methods
Magic methods" are special methods defined within a class that begin and end with double underscores.
For example, the __init__ method is a magic method responsible for initializing a newly created object.
Let's define a User class that overrides the magic method __init__. In this method, we will store the provided name and email in the class attributes. We will also define a method that returns the class attributes as a dictionary
class User:
def __init__(self, name, email):
self.name = name
self.email = email
def get_email_data(self):
return {
'name': self.name,
'email': self.email
}
adrian = User('Alexander', 'alexander@example.com')
print(adrian.get_email_data())
# Output: {'name': 'Alexander', 'email': 'alexander@example.com'}
Another magic method is the __new__ method, which defines what happens at the moment of creating an object of a class. The __new__ method returns the newly created object of the class.
For example, let's create a Singleton class that guarantees that no more than one object of this class can be created.
We can even try to create two objects, a and b, which will end up being the same object.
class Singleton:
instance = None
def __new__(cls):
if cls.instance is None:
cls.instance = super().__new__(cls)
return cls.instance
a = Singleton()
b = Singleton()
print(a is b) # Output: TrueThere is also the __del__ method, which defines behavior when an object is deleted. However, it doesn't always work in an obvious way. This method is not called when we delete an object using the del operator, but when the reference count of the object reaches zero and the garbage collector is triggered.
Of course, this doesn't always happen when we expect it to, which is why overriding the __del__ method is undesirable, as it can lead to memory leaks or unpredictable behavior.
Another magic method is the __str__ method. This method defines what happens when the print function is called on an object of the class.
The __str__ method should define a readable description of our class, which the user can later display in the interface.
In the following example, we use the previously written User class, but now if we call the print function on an object of the class, a clear and readable name will be displayed:
class User:
def __init__(self, name, email):
self.name = name
self.email = email
def __str__(self):
return f'{self.name} <{self.email}>'
adrian = User('Alexander', 'alexander@example.com')
print(adrian) # Output: Alexander <alexander@example.com>Two other useful magic methods are __hash__ and __eq__, which define what happens when the hash function is called and how objects are compared.
The __hash__ magic method overrides the hashing function, which is used, for example, when retrieving keys in a dictionary.
In the following example, we will specify in the User class that the hash function will use the user's e-mail for hashing, and users will be compared based on their e-mail addresses.
Thus, if we create two users with different names but the same e-mail addresses, Python will indicate that they are the same object when comparison is performed, because the overridden __eq__ method compares users by their e-mail addresses:
class User:
def __init__(self, name, email):
self.name = name
self.email = email
def __hash__(self):
return hash(self.email)
def __eq__(self, obj):
return self.email == obj.email
alex = User('Alex', 'alex@example.com')
alexander= User( 'Alexander', 'alex@example.com')
print(alex == alexander)
# Output: True
The hash() function now returns the same value because only the e-mail addresses are compared, and in this case, they are identical.
print(hash(alex))
# Output: -3795776245241034452
print(hash(alexander))
# Output: -3795776245241034452Very important magic methods are those that define access to attributes. These methods are __getattr__ and __getattribute__.
The __getattr__ method defines the behavior when an attribute we are trying to access is not found. The __getattribute__ method is called every time we access any attribute of an object.
Next, we will define a class and override the __getattribute__ method to always return the same string. As a result, no matter which attribute we access, the same string will be returned.
class Researcher:
def __getattr__(self, name):
return 'Nothing found :('
def __getattribute__(self, name):
return 'nope'
obj = Researcher()
print(obj.attr)
# Output: nope
print(obj.method)
# Output: nope
print(obj.DFG2H3J00KLL)
# Output: nopeThe __getattr__ method is called when an attribute is not found. In the following example, within __getattribute__, which is always called, we log that we are trying to access the corresponding attribute and then proceed using the object class. If the attribute is still not found, the __getattr__ method is invoked:
class Researcher:
def __getattr__(self, name):
return 'Nothing found :()\n'
def __getattribute__(self, name):
print(f'Looking for {name}')
return object.__getattribute__(self, name)
obj = Researcher()
print(obj.attr)
# Output: Looking for attr
# Nothing found :()
print(obj.method)
# Output: Looking for attr
# Nothing found :()
print(obj.DFG2H3J00KLL)
# Output: Looking for attr 21
# Nothing found :()The magic method __setattr__, as you might guess, defines the behavior when assigning a value to an attribute.
In this case, if we attempt to assign a value to an attribute, nothing will happen—the attribute will not be created.
class Ignorant:
def __setattr__(self, name, value):
print(f'Not gonna set {name}!')
obj = Ignorant()
obj.math = True
# Output: Not gonna set math!Finally, the __delattr__ method manages the behavior when deleting an attribute of an object. For example, it makes sense to use it if we want to cascade the deletion of objects related to the class.
In this case, we simply continue the deletion using the object class and log the fact that deletion is occurring.
class Polite:
def __delattr__(self, name):
value = getattr(self, name)
print(f'Goodbye {name}, you were {value}!')
object.__delattr__(self, name)
obj = Polite()
obj.attr = 10
del obj.attr
# Output: Goodbye attr, you were 10!Another magic method is __call__, which defines the behavior when the class is called as a function. For example, with the __call__ method, we can define a logger that we can then use as a decorator (yes, a decorator can be not only a function but also a class!).
In the example below, when initializing the Logger class, the object of this class remembers the filename provided to it. Each time the class is called, it returns a new function according to the decorator protocol and logs a line in the log file about the function call.
In this case, we define an empty function, and the decorator logs all its calls.
class Logger:
def __init__(self, filename):
self.filename filename
def __call__(self, func):
def wrapped (*args, **kwargs):
with open(self.filename, 'a') as f:
f.write('Test text...')
return func(*args, *kwargs)
return wrapped
logger = Logger('log.txt')
@logger
def completely_useless_function():
pass
completely_useless_function()
with open('log.txt') as f:
print(f.read())
# Output: Test text...In Python, the addition operation is handled by the __add__ operator (which is essentially a magic method). Similarly, subtraction can be overloaded using the __sub__ method.
As an example, let's define a class NoisyInt that behaves almost like an integer but adds noise during addition. To create random noise, we'll use the built-in random module.
import random
class NoisyInt:
def __init__(self, value):
self .value = value
def __add__(self, obj)
noise = random.uniform(-1, 1)
return self.value + obj.value + noise
a = NoisyInt(10)
b = Notsylnt(20)
for _ in range(3):
print(a + b)
# Output:
# 30.605646527205856
# 30.170967742734117
# 29.071231797981817
In Python, much of the magic when working with objects, classes, and methods is implemented using descriptors. To define your own descriptor, you need to implement the __get__, __set__, or __delete__ methods in a class.
After that, you can create a new class and assign an object of the descriptor type to an attribute of that class. This object will override the behavior for accessing the attribute (__get__), assigning values (__set__), or deleting the attribute (__delete__).
Let's create an object of the `Class` and see what happens when accessing an attribute:
class Descriptor:
def _get_ (self, obj, obj_type):
print('get')
def _set_ (self, obj, value):
print('set')
def _delete (self, obj):
print('delete')
class Class:
attr = Descriptor()
instance = Class()That's how the __get__ method works:
instance.attr
#Output: getAnd this is how the __delete__ method works:
del instance.attr
# Output: deleteAnd this is how the __set__ method works:
instance.attr = 10
# Output: setDescriptors are a powerful mechanism that allows you to override attribute behavior in classes transparently for the user (or another programmer using your code). For example, we can define a Value descriptor that overrides the behavior when assigning it a value.
Let’s define a class with an attribute that will be a descriptor, and we will observe the modified behavior (multiplying the value by 10) when assigning a value
class Value:
def __init__(self):
self.value = None
@staticmethod
def _prepare_value(value):
return value * 10
def __get__(self, obj, obj_type):
return self.value
def __set__(self, obj, value):
self.value = self._prepare_value(value)
class Class:
attr = Value()You can use this descriptor as follows:
obj = Class()
obj.attr = 5
print(obj.attr)
# Output: 50Even though you've been using functions and methods for a long time, you might not have realized that functions and methods are actually implemented using descriptors.
To understand that this is true, you can try accessing the same method through both an instance of the class and the class itself. It turns out that when you access a method via obj.method, it returns a bound method—a method tied to a specific object. But when you access the method from the class, Class.method, you get an unbound method—which is just a function.
This distinction highlights the role of descriptors in Python, where functions and methods are actually managed by the descriptor protocol to decide whether a method should be bound to an instance or left unbound.
As you can see, the same method returns different objects depending on how it is accessed. This is the behavior of a descriptor:
class Class:
def method(self):
pass
obj = Class()
print(obj.method)
# Output: <bound method Class.method of <_main_.Class object at 0x10ee77278>>
print(Class.method)
# Output: <function Class.method at 0x10ee3bea0>You might already be familiar with the @property decorator, which allows a function to be used as a class attribute. In this case, we define a full_name method, which, although it is a function returning a string, can be used just like a regular attribute—without parentheses when called.
When full_name is accessed from an instance of the class, it calls the full_name function. However, if you try to access full_name from the class itself, what you get is a property object.
In fact, property is implemented using descriptors, which explains the different behavior depending on how the object is accessed. Here is an example to illustrate this:
class User:
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
@property
def full_name(self):
return f'{self.first_name} {self.last_name}'
user= User('Alex', 'Misyuk')
print(user.full_name)
# Output: Alex Misyuk
print(User.full_name)
# Output: <property object at ....>To better understand how the property works, let's write our own class that mimics its behavior. We need to store the function that the property would normally receive.
When the attribute is accessed from the class, we'll just return self, and when the attribute is accessed from an instance, we'll call the stored function.
class Property:
def __init__(self, getter):
self.getter = getter
def __get__(self, obj, obj_type=None):
if obj is None:
return self
return self.getter(obj)Let's test our newly created decorator alongside the standard @property. To do this, we'll create another class Class and use the custom @MyProperty decorator. Here's how you can implement it:
class Class:
@property
def original(self):
return 'original'
@Property
def custom_sugar(self):
return 'custom sugar'
def custom_pure(self):
return 'custom pure'
custom_pure = Property(custom_pure)Here's what we ended up with
obj = Class()
print(obj.original)
# Output: original
print(obj.custom_sugar)
# Output: custom sugar
print(obj.custom_pure)
# Output: custom pureExactly the same way, @staticmethod and @classmethod are implemented using descriptors. Let's write our own implementation of these decorators.
For StaticMethod, we'll simply store the function and return it when accessed:
class StaticMethod:
def __init__(self, func):
self.func = func
def __get__(self, obj, obj_type=None):
return self.funcUnlike StaticMethod, ClassMethod returns a function where the first argument is obj_type, i.e., the class:
class ClassMethod:
def __init__(self, func):
self.func = func
def __get__(self, obj, obj_type=None):
if obj_type is None:
obj_type = type(obj)
def new_func(*args, **kwargs):
return self.func(obj_type, *args, **kwargs)
return new_funcAnother example of descriptors in the Python standard library is the __slots__ construct, which allows you to define a class with a fixed set of attributes.
Typically, each class has a dictionary where all its attributes are stored. However, this can often be excessive. If your code has a large number of objects, you might not want to create a dictionary for each object. In such cases, the __slots__ construct allows you to specify a fixed number of attributes that the class can have.
In the following example, we stipulate that our class should only have the attribute anakin.
class Class:
__slots__ = ['anakin']
def __init__(self):
self.anakin = 'the chosen one'If we try to add another attribute to the class, it won't work. To demonstrate this more clearly, let's use the interactive mode:
>>> obj = Class()
>>> obj.luke = 'the chosen two'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Class' object has no attribute 'luke'The __slots__ construct is implemented by defining descriptors for each of the attributes.
Metaclasses
As you already know, everything in Python is an object, and classes are no exception. This means that these classes are created by something. Let's define a class named Class and create an instance of it. The type of the object is Class because Class created it.
For clarity, let's use the interactive mode.
>>> class Class:
... pass
...
>>> obj = Class()
>>> type(obj)
<class '__main__.Class'>However, the class itself also has a type, specifically type, because type is the one that created the class. In this case, type is a metaclass, meaning it creates other classes.
>>> type(Class)
<class 'type'>It is very important to understand the difference between creation and inheritance. In this case, the class is not a subclass of type. type creates it, but the class does not inherit from type; instead, it inherits from the object class.
>>> issubclass(Class, type)
False
>>> issubclass(Class, object)
TrueTo understand how classes are defined, you can write a simple function that returns a class. In the following example, we define a function that returns a class. Classes can be created on the fly, and in this case, we create two different objects and return them.
def dummy_factory():
class Class:
pass
return Class
Dummy = dummy_factory()
Dummy() is Dummy()However, Python actually works differently. Classes are created using the metaclass type, and you can dynamically create a class by calling type and passing it the class name, a tuple of base classes, and a dictionary of attributes. For example, let's create a class NewClass without any parents or attributes.
This is a real class; we created it on the fly without using the class literal.
>>> NewClass = type('NewClass', (), {})
>>> NewClass
<class '__main__.NewClass'>
>>> NewClass()
<__main__.NewClass object at 0x7f29936ea908>Let's define our own metaclass Meta, which will manage the behavior during class creation. To be a metaclass, it must inherit from another metaclass (type). The metaclass method __new__ takes the class name, its parents, and attributes. Next, we can define a new class A and specify that its metaclass is Meta.
This metaclass will control the behavior when creating a new class. Thus, we can override the behavior of class creation (for example, add an attribute or do something else).
class Meta(type):
def __new__(cls, name, parents, attrs):
print(f'Creating {name}')
return super().__new__(cls, name, parents, attrs)
class A(metaclass=Meta):
pass
# Output: Creating AFor example, we can define a metaclass that overrides the __init__ method, so that every class created by this metaclass will keep track of all created subclasses. The new __init__ method records its own attribute, which will store a dictionary of created classes.
In the following example, we first create a class Base, with Meta as its metaclass. This class will have a class attribute registry, where we will store all its subclasses. Each time we create a class that inherits from Base, we record in registry the corresponding value, which includes the name of the created class and a reference to it.
class Meta(type):
def __init__(cls, name, bases, attrs):
print('Initializing {}'.format(name))
if not hasattr(cls, 'registry'):
cls.registry = {}
else:
cls.registry[name.lower()] = cls
super().__init__(name, bases, attrs)
class Base(metaclass=Meta):
pass
class A(Base):
pass
class B(Base):
pass
Comments (0)