Python type checking: Assertions, exceptions, Mypy

Second article in this series about typing support in Python will show you how to take type hints a step further and enforce type checking. Both Python 2 and 3 will be covered. You will also see why you may need a type support in the first place.

Python is based on duck-typing. But if your internal method works just with strings, the code will be more self-defensive, if it will accept only strings and not anything that can be potentially (and incorrectly) converted to a string. Swapped arguments is a good example.

Manual type checking

You can enforce type checking manually by certain calls or automatically by annotating your code with types and using a third party application for checking it. This section will show example of the manual approach.

Assertions

The easiest type checking can be done via assert  statements. Let us have a look at the following example from the previous article, this time with assertions:

This code has again a number of mistakes. blueberry_cake and  me have their arguments swapped, we are trying to bake appricot_cake, rather than a list of them and also dad.

When we try to run this code, it will fail on the first error:

As you can see, this is not very descriptive. Let us change the first assertion to:

This will produce much clearer error message:

It is always better to provide a description to asserts statements so it is immediately clear what is wrong. To illustrate why are assertions useful in the first place, let us try to remove them completely and run the code again:

What just happened? Do you know what to fix at first glance? The problem is that blueberry cake was happily created with wrong values and what failed was trying to use a string of "10" inside the time.sleep method. The sooner you catch an error, the easier it is to fix it.

Alternative string type matching in assert

There is a couple of alternatives to matching strings in isinstance:

  • assert isinstance(obj, str) - good for Python 3 as all strings are Unicode.
  • assert isinstance(obj, six.string_types) - will correctly work in both Python 2 and Python 3. Requires six to be installed.
  • assert isinstance(obj, typing.Text) - requires either typing to be installer or Python 3.5+.

Matching nested types in assert

In the bake method, we assert if cakes is list, tuple or set. But if someone provide a list of numbers, it is still wrong. Is there a way to match nested structures?

  • Define a method that performs the check, put it into assert - clear, reusable solution. May increase size of the code. If you need to do this, you should think of other solutions, such as Mypy.
  • assert all(map(lambda obj: isinstance(obj, Cake), cakes)) - unclear one-liner. Don't use it.
  • assert isinstance(cakes, typing.Iterable[Cake]) - while you may be tempted to use the typing module, this unfortunately does not work.

Exceptions

Exceptions can be used in a similar fashion as assertions. Let us rewrite our example so it will use exceptions instead:

You can see that we are less restrictive with exceptions. Rather that expecting specific types, we try to convert values first and we raise exceptions only when there is absolutely no way we could use the provided value.

When we run the code, only wrong_cake fails with the following error:

It is clear at first sight what is wrong with the code.

Assert or Exception?

If you are wondering, whether to use assertions or exceptions in your code, the answer is: both.

Use assertions for internal code and exceptions for public, interface library methods. This is why you never see an AssertionError exception when you use Python libraries.

There is a performance consideration as well. If you call your script as:

all assertions will be optimized away and never checked. This is acceptable for running your own code once it is tested but not for a library interface.

Duck typing

Duck typing tells us to expect only what we need to use. This is true for public, library methods. In the Exceptions example, the bake() method we check if cakes is instance of Iterable. Therefore,  bake() will accept any object that implements an __iter__()  method (this includes lists, tuples, sets and dicts). If you imagine this method is an interface to a library, it is a good idea to allow the library users use whatever can be iterated over and provide instances of Cake.

However, if we imagine bake() is an internally used method never exposed to any 3rd party, we most likely will know what data type we will use. Using a different type will usually mean an error because we are supplying something unexpected. In this case, we can be more restrictive and use the Assertions example.

Type hints provided by assert in PyCharm

Unfortunately, this time PyCharm will not pick up on the isinstance() uses from outside of the methods to provide type hinting. It will only use this information inside the methods. As you can see in the following example, PyCharm knows that name will be an instance of str and will show methods available for strings.

type checking: isinstance type hint in pycharm

Type checking by tools

In contract with the previous section, where you needed to explicitly do a type check via isinstance calls + assertions or exceptions, in the following section you will see an approach that make use of type annotated code a does these checks for you. You can find more about type annotation in the previous article from this series.

Mypy

Mypy is a static type checker for Python. You can think of also as a linter that checks proper type usage based on type-annotated code. Good news is that Mypy supports also partially annotated code and type annotations in comments. The documentation of Mypy provides enough of examples, so let me just show it in action. Let us use the example from the previous article that uses type annotations:

And let us run it through Mypy:

Most of the errors are detected by PyCharm as well. What Mypy detected and PyCharm did not is using "my cake" as one of the cakes to bake.

Conclusion

There are ways to provide type checking for every version of Python. Introducing type checks can increase your confidence in code and help catch bugs early. The beauty of using type checks in Python is that you can make the as restrictive as you need to. A method can accept only a specific type, a set of types, an abstract type or absolutely anything.

There is a danger of introducing unnecessary complexity, reduction of code clarity and slowing down the performance. If you can use type annotations and tool like Mypy, go for it. The overhead is minimal and performance overhead is none. For library functions, you will probably have to stick with Exceptions.

Additionally, ask your self how much restrictions you should impose. Library functions should be rather permissive, internal code more restrictive.

Související články / Related posts:

Zanechte komentář / Leave a comment

Your email address will not be published. Required fields are marked *