Second article in this series about typing sup­port in Python will show you how to take type hints a step further and enfor­ce type chec­king. Both Python 2 and 3 will be cove­red. You will also see why you may need a type sup­port in the first pla­ce.

Python is based on duck-typing. But if your inter­nal method works just with strings, the code will be more self-defensi­ve, if it will accept only strings and not any­thing that can be poten­ti­ally (and incorrect­ly) con­ver­ted to a string. Swap­ped argu­ments is a good exam­ple.

Manual type checking

You can enfor­ce type chec­king manu­ally by cer­ta­in calls or auto­ma­ti­cally by anno­ta­ting your code with types and using a thi­rd par­ty appli­cati­on for chec­king it. This secti­on will show exam­ple of the manu­al appro­ach.

Assertions

The easiest type chec­king can be done via assert  sta­te­ments. Let us have a look at the following exam­ple from the pre­vi­ous article, this time with asser­ti­ons:

import time


class Cake:
    def __init__(self, size, flavor):
        assert isinstance(size, (int, float))
        assert size > 0, "Size must be a positive number."
        assert isinstance(flavor, str)

        self._size = size
        self._flavor = flavor

    @property
    def size(self):
        return self._size


class Human:
    def __init__(self, name, likes):
        assert isinstance(name, str)
        assert isinstance(likes, (Cake, Human))

        self._name = name
        self._likes = likes


def bake(cakes):
    assert isinstance(cakes, (list, tuple, set))

    for cake in cakes:
        time.sleep(cake.size)


apricot_cake = Cake(10, "apricot")
blueberry_cake = Cake("blueberry", 10)

mum = Human("Mum", apricot_cake)
dad = Human("Dad", mum)
me = Human(blueberry_cake, "Radek")

bake([blueberry_cake])
bake(apricot_cake)
bake(dad)
bake(["my cake"])
bake([apricot_cake])

This code has aga­in a num­ber of mis­ta­kes. blueberry_cake and me have the­ir argu­ments swap­ped, we are try­ing to bake appricot_cake, rather than a list of them and also dad.

When we try to run this code, it will fail on the first error:

Traceback (most recent call last):
  File "test_assertions.py", line 45, in <module>
    blueberry_cake = Cake("blueberry", 10)
  File "test_assertions.py", line 9, in __init__
    assert isinstance(size, (int, float))
AssertionError

As you can see, this is not very descrip­ti­ve. Let us chan­ge the first asser­ti­on to:
assert isinstance(size, (int, float)), "Argument size must be int or float."

This will pro­du­ce much clearer error message:
Traceback (most recent call last):
  File "test_assertions.py", line 39, in <module>
    blueberry_cake = Cake("10", "blueberry")
  File "test_assertions.py", line 10, in __init__
    assert isinstance(size, (int, float)), "Argument size must be int or float."
AssertionError: Argument size must be int or float.

It is alwa­ys bet­ter to pro­vi­de a descrip­ti­on to asserts sta­te­ments so it is imme­di­a­te­ly clear what is wrong. To illustra­te why are asser­ti­ons use­ful in the first pla­ce, let us try to remo­ve them com­ple­te­ly and run the code aga­in:
Traceback (most recent call last):
  File "test_assertions.py", line 47, in <module>
    bake([blueberry_cake])
  File "test_assertions.py", line 35, in bake
    time.sleep(cake.size)
TypeError: an integer is required (got type str)

What just hap­pe­ned? Do you know what to fix at first glan­ce? The pro­blem is that blu­e­ber­ry cake was hap­pi­ly cre­a­ted with wrong valu­es and what fai­led was try­ing to use a string of „10” insi­de the time.sleep method. The soo­ner you catch an error, the easier it is to fix it.

Alternative string type matching in assert

The­re is a couple of alter­na­ti­ves to matching strings in isin­stan­ce:

  • assert isinstance(obj, str) – good for Python 3 as all strings are Uni­co­de.
  • assert isinstance(obj, six.string_types) – will correct­ly work in both Python 2 and Python 3. Requi­res six to be installed.
  • assert isinstance(obj, typing.Text) – requi­res either typing to be installer or Python 3.5+.

Matching nested types in assert

In the bake method, we assert if cakes is list, tuple or set. But if some­o­ne pro­vi­de a list of num­bers, it is still wrong. Is the­re a way to match nested structu­res?

  • Defi­ne a method that per­forms the check, put it into assert – clear, reu­sa­ble solu­ti­on. May incre­a­se size of the code. If you need to do this, you should think of other solu­ti­ons, such as Mypy.
  • assert all(map(lambda obj: isinstance(obj, Cake), cakes)) – unclear one-liner. Don’t use it.
  • assert isinstance(cakes, typing.Iterable[Cake]) – whi­le you may be tempted to use the typing modu­le, this unfor­tu­na­te­ly does not work.

Exceptions

Excep­ti­ons can be used in a simi­lar fashi­on as asser­ti­ons. Let us rewri­te our exam­ple so it will use excep­ti­ons inste­ad:

import time
import collections.abc


class Cake:
    def __init__(self, size, flavor):
        self._size = float(size)

        if self._size <= 0:
            raise ValueError("Argument size must be a positive number.")

        self._flavor = str(flavor)

    @property
    def size(self):
        return self._size


class Human:
    def __init__(self, name, likes):
        if not isinstance(likes, (Cake, Human)):
            raise TypeError("Argument likes must be Cake or Human.")

        self._name = str(name)
        self._likes = likes


def bake(cakes):
    if not isinstance(cakes, collections.abc.Iterable):
        raise TypeError("Argument cakes must Iterable.")

    for cake in cakes:
        time.sleep(cake.size)


apricot_cake = Cake(10, "apricot")
blueberry_cake = Cake("10", "blueberry")
wrong_cake = Cake("apricot", 10)

You can see that we are less rest­ricti­ve with excep­ti­ons. Rather that expecting spe­ci­fic types, we try to con­vert valu­es first and we rai­se excep­ti­ons only when the­re is abso­lu­te­ly no way we could use the pro­vi­ded value.

When we run the code, only wrong_cake fails with the following error:

Traceback (most recent call last):
  File "test_exceptions.py", line 39, in <module>
    wrong_cake = Cake("apricot", 10)
  File "test_exceptions.py", line 8, in __init__
    self._size = float(size)
ValueError: could not convert string to float: 'apricot'

It is clear at first sight what is wrong with the code.

Assert or Exception?

If you are won­de­ring, whe­ther to use asser­ti­ons or excep­ti­ons in your code, the answer is: both.

Use asser­ti­ons for inter­nal code and excep­ti­ons for pub­lic, inter­fa­ce lib­ra­ry methods. This is why you never see an Asser­ti­o­nError excep­ti­on when you use Python lib­ra­ries.

The­re is a per­for­man­ce con­si­de­rati­on as well. If you call your script as:

python -O script.py

all asser­ti­ons will be opti­mi­zed away and never chec­ked. This is acceptable for run­ning your own code once it is tes­ted but not for a lib­ra­ry inter­fa­ce.

Duck typing

Duck typing tells us to expect only what we need to use. This is true for pub­lic, lib­ra­ry methods. In the Excep­ti­ons exam­ple, the bake() method we check if cakes is instan­ce of Ite­ra­ble. The­re­fo­re, bake() will accept any object that imple­ments an __iter__()  method (this inclu­des lists, tuples, sets and dicts). If you ima­gi­ne this method is an inter­fa­ce to a lib­ra­ry, it is a good idea to allow the lib­ra­ry users use wha­te­ver can be ite­ra­ted over and pro­vi­de instan­ces of Cake.

However, if we ima­gi­ne bake() is an inter­nally used method never expo­sed to any 3rd par­ty, we most like­ly will know what data type we will use. Using a dif­fe­rent type will usu­ally mean an error because we are sup­ply­ing some­thing unex­pec­ted. In this case, we can be more rest­ricti­ve and use the Asser­ti­ons exam­ple.

Type hints provided by assert in PyCharm

Unfor­tu­na­te­ly, this time PyCharm will not pick up on the isin­stan­ce() uses from out­si­de of the methods to pro­vi­de type hin­ting. It will only use this infor­mati­on insi­de the methods. As you can see in the following exam­ple, PyCharm knows that name will be an instan­ce of str and will show methods avai­la­ble for strings.

type checking: isinstance type hint in pycharm

Type checking by tools

In con­tract with the pre­vi­ous secti­on, whe­re you nee­ded to expli­cit­ly do a type check via isin­stan­ce calls + asser­ti­ons or excep­ti­ons, in the following secti­on you will see an appro­ach that make use of type anno­ta­ted code a does these checks for you. You can find more about type anno­tati­on in the pre­vi­ous article from this series.

Mypy

Mypy is a sta­tic type chec­ker for Python. You can think of also as a lin­ter that checks pro­per type usage based on type-anno­ta­ted code. Good news is that Mypy sup­ports also par­ti­ally anno­ta­ted code and type anno­tati­ons in com­ments. The docu­men­tati­on of Mypy pro­vi­des enou­gh of exam­ples, so let me just show it in acti­on. Let us use the exam­ple from the pre­vi­ous article that uses type anno­tati­ons:

import time
import typing


class Cake:
    def __init__(self, size: int, flavor: str) -> None:
        """
        :param size: Size of the cake.
        :param flavor: Flavor of the cake.
        """
        self._size = size
        self._flavor = flavor

    @property
    def size(self) -> int:
        return self._size


class Human:
    def __init__(self, name: str, likes: typing.Union[Cake, 'Human']) -> None:
        self._name = name
        self._likes = likes


def bake(cakes: typing.List[Cake]) -> None:
    for cake in cakes:
        time.sleep(cake.size)


apricot_cake = Cake(10, "apricot")
blueberry_cake = Cake("10", "blueberry")
wrong_cake = Cake("apricot", 10)

mum = Human("Mum", apricot_cake)
dad = Human("Dad", mum)
me = Human(blueberry_cake, "Radek")

bake(apricot_cake)
bake(dad)
bake(["my cake", apricot_cake])
bake([apricot_cake])

And let us run it throu­gh Mypy:
$ mypy test_annotations.py 
test_annotations.py:31: error: Argument 1 to "Cake" has incompatible type "str"; expected "int"
test_annotations.py:32: error: Argument 1 to "Cake" has incompatible type "str"; expected "int"
test_annotations.py:32: error: Argument 2 to "Cake" has incompatible type "int"; expected "str"
test_annotations.py:36: error: Argument 1 to "Human" has incompatible type "Cake"; expected "str"
test_annotations.py:36: error: Argument 2 to "Human" has incompatible type "str"; expected "Union[Cake, Human]"
test_annotations.py:38: error: Argument 1 to "bake" has incompatible type "Cake"; expected List[Cake]
test_annotations.py:39: error: Argument 1 to "bake" has incompatible type "Human"; expected List[Cake]
test_annotations.py:40: error: List item 0 has incompatible type "str"

Most of the errors are detec­ted by PyCharm as well. What Mypy detec­ted and PyCharm did not is using „my cake” as one of the cakes to bake.

Conclusion

The­re are ways to pro­vi­de type chec­king for eve­ry ver­si­on of Python. Intro­du­cing type checks can incre­a­se your con­fi­den­ce in code and help catch bugs ear­ly. The beau­ty of using type checks in Python is that you can make the as rest­ricti­ve as you need to. A method can accept only a spe­ci­fic type, a set of types, an abs­tract type or abso­lu­te­ly any­thing.

The­re is a dan­ger of intro­du­cing unne­cessa­ry com­ple­xi­ty, reducti­on of code cla­ri­ty and slowing down the per­for­man­ce. If you can use type anno­tati­ons and tool like Mypy, go for it. The overhe­ad is mini­mal and per­for­man­ce overhe­ad is none. For lib­ra­ry functi­ons, you will pro­ba­bly have to stick with Excep­ti­ons.

Addi­ti­o­nally, ask your self how much rest­ricti­ons you should impose. Lib­ra­ry functi­ons should be rather per­mis­si­ve, inter­nal code more rest­ricti­ve.


0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.