Finding Magic in Python

Anna-Lena Popkes

October 8, 2020

About Me

Machine Learning Engineer @inovex

About Me

Member of “KI Macht Schule”

About Me

About Me

Github: zotroneneis

The Tales of Castle Kilmere

Characters

Characters

Caste Kilmere

Headmistress

Faculties

Caste Kilmere

Dark Army

Classes

Object oriented programming (OOP)

  • Programs contain objects that interact with each other
  • Objects can contain attributes and methods
  • OOP represents the structure of the real world

What is a class?

  • Acts as a blueprint for an object
  • Describes how members of a class are structured, and which attributes and methods they have

Creating a class

class CastleKilmereMember:
    pass

kilmere_member = CastleKilmereMember()

Adding attributes and methods

class CastleKilmereMember:
    def __init__(self, name, birthyear, sex):
        self.name = name
        self.birthyear = birthyear
        self.sex = sex
        
    def says(self, words):
        return f"{self.name} says: {words}"

Adding attributes and methods

class CastleKilmereMember:
    def __init__(self, name, birthyear, sex):
        ...

miranda = CastleKilmereMember('Miranda Mirren',
                              1962,
                              'female')

print(miranda.says("Hello my dear"))
>>> Miranda Mirren says: Hello my dear

__init__

  • Called when creating a new instance of a class
  • The first argument of __init__ is self
  • self points towards an instance of the class

Inheritance

  • Problem: We want more classes (pupils, professors, ghosts, …)
  • All of these are members of Castle Kilmere
  • This is what inheritance is used for
  • Allows us to create a new class that inherits all attributes/methods from the parent class
  • The child class can override attributes/methods of the parent class and add new functionality

Inheritance

class Pupil(CastleKilmereMember):
    def __init__(self, name, birthyear, sex, start_year, pet=None):
        super().__init__(name, birthyear, sex)
        self.start_year = start_year

        if pet is not None:
            self.pet_name, self.pet_type = pet

        self._elms = {
                  'Critical Thinking': False,
                  'Self-Defense Against Fresh Fruit': False,
                  'Broomstick Flying': False,
                  'Magical Theory': False,
                  'Foreign Magical Systems': False,
                  'Charms': False,
                  'Defence Against Dark Magic': False,
                  'History of Magic': False,
                  'Potions': False,
                  'Transfiguration': False}

Inheritance

lissy = Pupil(name='Lissy Spinster',
              birthyear=2008,
              sex='female',
              start_year=2020,
              pet=('Ramses', 'cat'))

Summary

  • In Python, classes implement the OOP paradigm
  • Classes act as a blueprint for an object
  • Parent class: CastleKilmereMember
  • Child class: Pupil

Types of class methods

Types of class methods

  1. Instance methods
  2. Class methods
  3. Static methods

Instance methods

  • Most common type of method
  • At least one input parameter (self)
  • Can modify both object state and class state

Instance methods

Our base class already has an instance method:

class CastleKilmereMember:
    def __init__(self, name, birthyear, sex):
        ...
        
    def says(self, words):
        return f"{self.name} says {words}"

Class methods

  • Take at least cls as an input
  • cls points towards the class
  • Can modify class state but not object state
  • Applied using the @classmethod decorator

Note

The names self and cls are only conventions

Alternative constructors

  • A class can only have one constructor (__init__)
  • We can use class methods to create additional constructors

Alternative constructors

class Pupil(CastleKilmereMember):
   ...
      
   @classmethod
   def lissy(cls):
       return cls('Lissy Spinster',
                   2008,
                  'female',
                   2018,
                  ('Ramses', 'cat'))

lissy = Pupil.lissy()

Static methods

  • Take neither self nor cls as an input
  • Cannot modify object state or class state
  • Related to the class but yet independent
  • Can only access data they are provided with

Static methods

class Pupil(CastleKilmereMember):
    ...

    @staticmethod
    def passed(grade):
        grades = {
                'E': True,
                'Excellent': True,
                'O': True,
                'Ordinary': True,
                'A': True,
                'Acceptable': True,
                'P': False,
                'Poor': False,
                'H': False,
                'Horrible': False,
                }

        return grades[grade]

Static methods

lissy = Pupil.lissy()
luke = Pupil.luke()

print(lissy.passed('O'))
>>> True
print(luke.passed('H'))
>>> False
print(Pupil.passed('O'))
>>> True

Why do we need class/static methods?

  • They allow developers to communicate their intention
  • Example: static method expresses independence
  • Class methods can be used as alternative constructors

Summary

  • Three types of methods: instance, class and static methods
  • Note: using static methods is controversial

Type annotations

What are type annotations?

  • Allow adding arbitrary metadata to function arguments and return value
  • Document the types of parameters and return value
  • Allow for automated type checking

Syntax

class CastleKilmereMember:
    def __init__(self, name: str, birthyear: int, sex: str):
        ...

class Professor(CastleKilmereMember):
    def __init__(self, name: str, birthyear: int, sex: str,
                 subject: str):
        ...

Specifying the return value

class CastleKilmereMember:
    def __init__(self, name: str, birthyear: int, sex: str):
        self.name = name
        self.birthyear = birthyear
        self.sex = sex
        
    def says(self, words: str) -> str:
        return f"{self.name} says: {words}"

Referencing classes

class CastleKilmereMember:
    def __init__(self, name: str, birthyear: int, sex: str):
        self.name = name
        self.birthyear = birthyear
        self.sex = sex
        
    @classmethod
    def school_headmistress(cls) -> 'CastleKilmereMember':
        return cls('Miranda Mirren',
                    1963,
                   'female')

Advanced type hints

from typing import Tuple
class Pupil(CastleKilmereMember):
    def __init__(..., pet: Tuple[str, str]):
           ...

    @classmethod
    def lissy(cls) -> 'CastleKilmereMember':
        return cls('Lissy Spinster',
                    2008,
                   'female',
                    2018,
                   ('Ramses', 'cat'))

Summary

Type annotations are awesome!

To-string conversion

Motivation

Printing an object does not give a useful representation

bromley = CastleKilmereMember('Bromley Huckabee',
                               1959,
                              'male')
print(bromley)

Motivation

Printing an object does not give a useful representation

bromley = CastleKilmereMember('Bromley Huckabee',
                               1959,
                              'male')
print(bromley)
>>> <__main__.CastleKilmereMember object at 0x7f81853bfc50>

Controlling to-string conversion

  • There are 2 methods that control how object are converted to strings
    • __repr__
    • __str__
  • They have different purposes

__repr__

The output should be as unambiguous as possible

__repr__

import datetime
now = datetime.datetime.now()
repr(now) # What is the output?

__repr__

import datetime
now = datetime.datetime.now()
repr(now) 
>>> datetime.datetime(2020, 9, 29, 16, 7, 23, 941191)

__repr__

import datetime
now = datetime.datetime.now()

now == eval(repr(now))
>>> True

__str__

The output should be readable

__str__

import datetime
now = datetime.datetime.now()
str(now) 
>>> 2020-09-27 12:29:01.366385

__repr__ or __str__?

  • Define at least __repr__
  • Ensures a useful conversion of objects to strings
  • Is used as a fallback if __str__ is not implemented

Example

class CastleKilmereMember:
    def __init__(self, name: str, birthyear: int, sex: str):
        ...

    def __repr__(self) -> str:
        return (f"{self.__class__.__name__}(name='{self.name}', "
                f"birthyear={self.birthyear}, sex='{self.sex}')")

Example

# Old result of print(bromley)
# <__main__.CastleKilmereMember object at 0x7f81853bfc50>

# New result
print(bromley)
>>> CastleKilmereMember(name='Bromley Huckabee',
                        birthyear=1959, sex='male')

bromley == eval(repr(bromley))
>>> True

Summary

  • To-string conversion of objects is controlled by __repr__ and __str__
  • Implement at least __repr__

defaultdict

The collections module

  • Contains several useful classes
  • Especially helpful for the magical universe: collections.defaultdict

Extending CastleKilmereMember

class CastleKilmereMember:
    def __init__(self, ...):
        ...
        self._traits = {}

    def add_trait(self, trait: str, value: bool = True):
        self._traits[trait] = value

    def print_traits(self):
        ...

bromley = CastleKilmereMember(
        'Bromley Huckabee', 1959, 'male'
        )
bromley.add_trait("kind")
bromley.add_trait("tidy-minded")
bromley.add_trait("impatient", value=False)

bromley.print_traits()
>>> Bromley Huckabee is kind and tidy-minded
>>> Bromley Huckabee is not impatient

Task: add method that checks if a Castle Kilmere member exhibits a certain character trait

class CastleKilmereMember:
    def __init__(self, ...):
        ...
        self._traits = {}

    def exhibits_trait(self, trait: str) -> bool:
        ...

CastleKilmereMember

  • exhibits_trait should return True if a trait exists and False if it doesn’t
  • Solution one: dict.get()
  • Solution two: collections.defaultdict

dict.get()

class CastleKilmereMember:

    def __init__(self, ...):
        ...
        self._traits = {}

    def add_trait(self, trait: str, value=True):
        self._traits[trait] = value
            
    def exhibits_trait(self, trait: str) -> bool:
        value = self._traits.get(trait, False)
        return value

collections.defaultdict

  • Subclass of the general dictionary type
  • Allows us to specify a callable whose return value will be used for missing items

Basic usage

from collections import defaultdict
my_dict = defaultdict(default_factory)

collections.defaultdict

  • Requires a callable default_factory as an argument
  • The default value is returned whenever a requested key cannot be found

CastleKilmereMember

  • Goal: return False as a default value
  • Maybe dict_ = defaultdict(False)?
  • Why is this wrong?

CastleKilmereMember

  • defaultdict requires a callable as an argument
  • False is not callable
  • We have to define a function that returns False when called without arguments

CastleKilmereMember

def return_false() -> bool:
    return False

dict_ = defaultdict(return_false)

CastleKilmereMember

Alternative:

dict_ = defaultdict(lambda: False)

The power of defaultdict

defaultdict can be provided with any kind of callable

Use case - grouping items

from collections import defaultdict

pets = [('Cotton', 'owl'), ('Ramses', 'cat'),
        ('Twiggles', 'owl'), ('Oscar', 'cat'),
        ('Louie', 'cat'), ('Bob', 'ferret'),
        ('Winston', 'owl'), ('Harry', 'owl')]

Use case - grouping items

from collections import defaultdict

pets = [('Cotton', 'owl'), ('Ramses', 'cat'),
        ('Twiggles', 'owl'), ('Oscar', 'cat'),
        ('Louie', 'cat'), ('Bob', 'ferret'),
        ('Winston', 'owl'), ('Harry', 'owl')]

types_of_pets = defaultdict(list)
for name, type_ in pets:
    types_of_pets[type_].append(name)

# What is the output of the following?
for key, value in types_of_pets.items():
    print(f"{key}: {value}")

for key, value in types_of_pets.items():
    print(f"{key}: {value}")

>>> owl: ['Cotton', 'Twiggles', 'Winston', 'Harry']
>>> cat: ['Ramses', 'Oscar', 'Louie']
>>> ferret: ['Bob']

Summary

  • Common problem: accessing keys in a dictionary that don’t exist
  • Can be handled with collections.defaultict
  • Behaves nearly identical to a regular Python dictionary
  • Difference: keys are created and populated with a default value when trying to access or modify a missing key

Decorators

What are decorators?

  • Short answer: callable that takes a callable as an input and returns a callable
  • Allow us to extend and/or modify the behavior of the input callable
  • The decorated callable is not permanently modified

The simplest decorator returns its input function

def useless_decorator(function):
    return function

We apply a decorator to a function by wrapping it

def say_hello() -> str:
    return f"Hey there!"

say_hello = useless_decorator(say_hello)

Alternative syntax

@useless_decorator
def say_hello() -> str:
    return f"Hey there!"

Modifying the wrapped function’s behavior

  • To modify behavior a decorator has to define a wrapper function
  • This wraps the input function and modifies its behavior

Example

def goodbye(function):
    def wrapper():
        original_output = function()
        new_output = original_output +
                     f" Goodbye, have a good day!"
        return new_output
    return wrapper

@goodbye
def say_hello() -> str:
    return f"Hey there!"

# What is the output of this print statement?
print(say_hello())

Example

def goodbye(function):
    def wrapper():
        original_output = function()
        new_output = original_output +
                     f" Goodbye, have a good day!"
        return new_output
    return wrapper

@goodbye
def say_hello() -> str:
    return f"Hey there!"

print(say_hello())
>>> Hey there! Goodbye, have a good day!

Functions with input arguments

def say_words(person: str, words: str) -> str:
    return f"{person} says: {words}"

print(say_words("Lissy", "Hey Luke!"))
>>> Lissy says: Hey Luke!

Functions with input arguments

  • How can we decorate this function?
  • The goodbye wrapper function must be able to process the inputs person and words
  • Solution: use *args and **kwargs

Functions with input arguments

def goodbye(function):
    def wrapper(*args, **kwargs):
        original_output = function(*args, **kwargs)
        new_output = original_output +
                     f" Goodbye, have a good day!"
        return new_output
    return wrapper

@goodbye
def say_words(person: str, words: str) -> str:
    return f"{person} says: {words}"

# What is the output of this print statement?
print(say_words("Lissy", "Hey Luke!"))

print(say_words("Lissy", "Hey Luke!"))
>>> Lissy says: Hey Luke! Goodbye, have a good day!

Why are decorators called decorators?

Because they “decorate” other functions and allow us to run code before and after the wrapped function is executed.

Summary

  • Decorators allow us to modify the behaviour of a function
  • The decorated function only changes when it’s decorated
  • Decorators are a complex topic and we have only scratched the surface

Abstract Base Classes

Magical universe

  • Parent class (CastleKilmereMember)
  • Several child classes (Pupil, Professor, etc.)
  • Child classes inherit all methods from their parent class
  • In some cases simple inheritance is not sufficient

Abstract base classes (ABCs)

  • Useful if an application involves a hierarchy of classes
  • In this hierarchy:
    • It should be impossible to instantiate the base class
    • All subclasses should have a common base class
    • All subclasses should implement certain methods defined in the base class

Example

Example

from abc import ABC, abstractmethod

class Spell(ABC):
    def __init__(self, name: str, incantation: str, effect: str):
        self.name = name
        self.incantation = incantation
        self.effect = effect

    @abstractmethod
    def cast(self):
        pass

    @property
    @abstractmethod
    def defining_feature(self):
        pass

Introspection

Spell.__abstractmethods__
>>> frozenset({'cast', 'defining_feature'}) 

Can we instantiate the Spell class?

stuporus = Spell(name='The stuporus ratiato spell',
                 incantation='Stuporus Ratiato',
                 effect='Makes objects fly')

stuporus = Spell(name='The stuporus ratiato spell',
                 incantation='Stuporus Ratiato',
                 effect='Makes objects fly')

>>> TypeError: Can't instantiate abstract class Spell with 
    abstract methods cast, defining_feature

Subclass Charm

class Charm(Spell):
    def __init__(self, name: str, incantation: str,
                 effect: str, difficulty: str = "Simple",
                 min_year: int = 1):
        super().__init__(name, incantation, effect)
        self.difficulty = difficulty
        self.min_year = min_year

    def cast(self):
        print(f"{self.incantation}!")

Can we instantiate the Charm class?

stuporus = Charm(name="The stuporus ratiato spell",
                 incantation="Stuporus Ratiato",
                 effect="Makes objects fly",
                 difficulty="Simple")

stuporus = Charm(name="The stuporus ratiato spell",
                 incantation="Stuporus Ratiato",
                 effect="Makes objects fly",
                 difficulty="Simple")

>>> TypeError: Can't instantiate abstract class Charm
    with abstract methods defining_feature

But why?

Answer

Because we forgot to implement the defining_feature method!

ABC vs. normal class

  • Situation: a subclass does not implement all methods required by the base class
  • When does a normal class raise an error?
  • Answer: only when calling the missing method
  • When does an ABC raise an error?
  • Answer: at instantiation time

class Charm(Spell):
    def __init__(self, name: str, incantation: str,
                 effect: str, difficulty: str = "Simple",
                 min_year: int = 1):
        super().__init__(name, incantation, effect)
        self.difficulty = difficulty
        self.min_year = min_year

    @property
    def defining_feature(self) -> str:
        return ("Alteration of the object's inherent
                 qualities, that is, its behaviour
                 and capabilities")

    def cast(self):
        print(f"{self.incantation}!")

stuporus = Charm(name="The stuporus ratiato spell",
                 incantation="Stuporus Ratiato",
                 effect="Makes objects fly",
                 difficulty="Simple")

print(stuporus)
>>> Charm(name="The stuporus ratiato spell",
          incantation='Stuporus Ratiato',
          effect='Makes objects fly',
          difficulty='Simple')

Stacking decorators

@property
@abstractmethod
def defining_feature(self):
    pass

How can we express this in the ‘wrapping’ syntax?

Stacking decorators

defining_feature = property(abstractmethod(defining_feature))

Stacking decorators

When stacking decorators abstractmethod() should be applied as the innermost decorator

Summary

  • ABCs formalize the relationship between a parent class and a subclass
  • Three purposes:
    • Allow the parent class to demand a certain structure of their subclasses
    • Allow subclasses to identify as meeting those requirements
    • Enforce that a subclass meets the requirements
  • ABCs are a huge topic - we have only seen a small part of it

Namedtuples

What is a tuple?

Tuples

  • Data structure for grouping arbitrary objects
  • Tuples are immutable

Tuples

The pet attribute of the Pupil class is a tuple:

pet = ('name', 'type')
lissys_pet = ('Ramses', 'cat')

Tuples

lissys_pet = ('Ramses', 'cat')
print(lissys_pet[0])
>>> Ramses

Tuples are immutable

lissys_pet[0] = 'Twiggles'
>>> TypeError: 'tuple' object does not support
    item assignment

Namedtuples

  • Variation of plain tuples
  • Allow us to name the fields of the tuple
  • Easier to access individual fields
  • Makes code more readable

Creating namedtuples

  • collections.namedtuple
  • typing.NamedTuple

typing.NamedTuple

  • Allows us to specify the type of each field
  • Makes it easy to add methods to the class

typing.NamedTuple

from typing import NamedTuple

class Pet(NamedTuple):
    name: str
    type: str

lissys_pet = Pet('Ramses', 'cat')

print(lissys_pet)
>>> Pet(name='Ramses', type='cat')

Accessing fields

name = lissys_pet[0]
# or
name = lissys_pet.name

Magical universe

  • Pupils, professors and ghosts should not be immutable
  • Suitable group of people: Dark Army members

Dark Army

Dark Army class

from typing import NamedTuple

class DarkArmyMember(NamedTuple):
    name: str
    birthyear: int

    @classmethod
    def leader(cls) -> 'DarkArmyMember':
        return cls('Master Odon', 1971)

Dark Army class

From now on we can easily create new Dark Army members:

keres = DarkArmyMember('Keres Fulford', 1983)

print(keres)
>>> DarkArmyMember(name='Keres Fulford', birthyear=1983)

print(keres.leader())
>>> DarkArmyMember(name='Master Odon', birthyear=1971)

keres.name = "Mortimer"
>>> AttributeError: can't set attribute

Summary

  • Namedtuples are an extension of plain tuples
  • They present a shortcut for creating immutable classes
  • With Python 3.7 we can also use data classes for creating immutable classes

Data Classes

What are data classes?

  • PEP: “mutable namedtuples with defaults”
  • Implement several useful methods by default

Creating a data class

from dataclasses import dataclass

@dataclass
class Department:
    name: str
    head: Professor
    founded_in: int
    ghost: Ghost

Creating instances

briddle = Professor.briddle()
mocking_knight = Ghost.mocking_knight()
law_department = Department(name='Department of Law',
                            head=briddle,
                            founded_in=785,
                            ghost=mocking_knight)

Default functionality

  • Several special methods are automatically added
  • Example: __init__():
def __init__(self, name: str, head: Professor,
             founded_in: int, ghost: Ghost):
    self.name = name
    self.head = head
    self.founded_in = founded_in
    self.ghost = ghost

Also __repr__() is added automatically

print(law_department)
>>> Department(name='Department of Law',
               head=Professor(
                   name='Birdie Briddle', 
                   birthyear=1931, 
                   subject='Foreign Magical Systems'
               ),
               ghost=Ghost(
                   name='The Mocking Knight', 
                   birthyear=1401, 
                   sex='male',
                   year_of_death=1492
               ), 
               founded_in=785)

Adding default values

from dataclasses import dataclass

@dataclass
class Department:
    name: str
    head: Professor
    founded_in: int = 785
    ghost: Ghost

Adding methods

import datetime

@dataclass
class Department:
    name: str
    head: Professor
    founded_in: int = 785
    ghost: Ghost

    def calc_age_in_years(self) -> int:
        now = datetime.datetime.now().year
        return now - self.founded_in

Providing parameters

  • We can provide parameters when using the @dataclass decorator
  • Example parameters:
    • init: If true, __init__() will be generated
    • repr: If true, __repr__() will be generated
    • frozen: If true, fields are frozen so assigning to fields will raise an exception

Immutable Data Classes

from typing import NamedTuple

class DarkArmyMember(NamedTuple):
    name: str
    birthyear: int

    @classmethod
    def leader(cls) -> 'DarkArmyMember':
        return cls('Master Odon', 1971)

Immutable Data Classes

@dataclass(frozen=True)
class DarkArmyMember:
    name: str
    birthyear: int

    @classmethod
    def leader(cls) -> 'DarkArmyMember':
        return cls('Master Odon', 1971)

Immutable Data Classes

keres = DarkArmyMember('Keres Fulford', 1983)
print(keres)
>>> DarkArmyMember(name='Keres Fulford', birthyear=1983)

print(keres.leader())
>>> DarkArmyMember(name='Master Odon', birthyear=1971)

keres.name = "Mortimer"
>>> FrozenInstanceError: cannot assign to field 'name'

Summary

  • Data classes simplify creating classes that store a lot of state
  • They can be configured using @dataclass with parameters
  • They can be made immutable

THE END

// reveal.js plugins