Fundamentals of Type Hints in Python

Part 1: Basics of type hints in Python.

Photo by Arnold Antoo on Unsplash

Welcome to this series of posts where we will explain from the basics to the most advanced use cases of type annotations in Python. In this first chapter, we will see the fundamental concepts that will be used as a basis for the following chapters, as well as some concepts necessary to understand the philosophy of Python.

Have you ever gone back to look at your code, and don't remember what type of argument expect some function?. Sometimes you have a linter that can give you some hints about the methods and properties of some objects but the dynamic typing of Python doesn't allow you to infer this information.

Python type hints are a big change in the python ecosystem, through them is possible to annotate the types of variables, and functions and even create new types! With the help of type annotations, the IDEs have access to all the information necessary to give you some feedback about the property, method, and possible values that can have the variable in your source code, all of this without dropping all your existed code.

The goal of type hints is to allow the programmer to add annotations of types in the part of the system that required it, and allow the rest of the code to work as before.

It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.

Guido van Rossum, Jukka Lehtosalo, and Łukasz Langa, PEP 484 — Type Hints

The problem

During software development using Python, is a common problem to not know exactly what type of variable we have. Some alternatives to avoid this problem are the annotation of the expected types in the function/method docstring, for example:

def sum(a, b):
  '''
  args:
    a: number type (float, complex, int)
    b: number type (float, complex, int)
  return:
    number that is the sum of a and b
  '''
  return a + b

A docstring is A string literal that appears as the first expression in a class, function or module. While ignored when the suite is executed, it is recognized by the compiler and put into the __doc__ attribute of the enclosing class, function or module. Since it is available via introspection, it is the canonical place for documentation of the object.

Python glossary: Definition of docstring

Note: An interesting note is that Python docstrings by default are kept around indefinitely in memory since they’re accessible via the __doc__ attribute of a function or a module. The -OO option to the interpreter causes it to remove docstrings from the generated .pyo and avoid this behavior.

This is a convention that allows, both the users and the creators of the function to know the expected type for function arguments, but this has one additional problem, this docstring is not interpreted by many IDE, then some cool features like autocomplete methods, linter errors, etc, are not always available.

One extra problem when this approach, is that the documentation can be outdated in time. If someone updates the function and forgets to add the proper documentation, then there is not enough information for inferring the new changes.

Finally, another problem is that not exists an official convention for this kind of documentation, this may vary depending on the team, or the personal preference.

In the end, this strategy expects that the user of the function is capable of getting the function docstring in a fast way. This may be a fact of life in modern IDEs, but not exists an automated program that can run your program and assert that you pass the correct types of variables according to this docstring, is only a “virtual convention” between the user and the creator of the function.

Why not force the right type?

Another alternative is to try to secure the type of arguments at the beginning of the function but this is a slow and tedious process.

def f(n):
  if not isinstance(n, int):
    raise ValueError(f'the n parameter must to be a int, but instead is {type(n)}')

So, what is the solution?

— The solution is Type hints.

Duck typing

— If it walks like a duck and it quacks like a duck, then it must be a duck

Before delving into Type hints, is important to know what is the foundations of Python, and what is the idiomatic way of writing Python. To refresh your knowledge, I going to introduce the concept of Duck typing.

There are many definitions of the concept of type in the literature. Here we assume that type is a set of values and a set of functions that one can apply to these values.

PEP 483: The Theory of Type Hints

According to the Python glossary, duck typing is:

A programming style that does not look at an object’s type to determine if it has the right interface; instead, the method or attribute is simply called or used (“If it looks like a duck and quacks like a duck, it must be a duck.”) By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). (Note, however, that duck-typing can be complemented with abstract base classes.) Instead, it typically employs hasattr() tests or EAFP programming.

Python glossary: Duck typing definition

Duck typing is one of the core ideas of Python, it’s about the concept of using the structure of the class (methods) and not the type of the class, sometimes this is referred to as design by capabilities rather than design by contract.

For example, imagine a situation where you have a Game system that has many objects in the scene. Some of that objects are of type Person and others of type Duck.

In this case, the system generates random (or intelligent) moves in each second in order to generate a more interactive environment.

class Person:
  def walk(self, new_position):
    ...

class Duck:
  def walk(self, new_position):
    ...

class GameSystem:
    # ...
    def register_obj(self, obj):
       self.scene.append(obj)
    
    def refresh_environment(self):
        # Executed each second
        for obj in self.scene:
            random_position = ...
            obj.walk(random_position)

If you pay attention to the Person and Duck class, they do not share some parent class or implement a common interface in an explicit way, however, both have the same capability of walking.

After that, GameSystem has a refresh_environment method responsible for calling the method walk in all the objects in the system. And a question that might come to mind is, what kind of objects do you have? , the reality is that it does not matter, while objects have the capability of walking, these objects can be of any type.

Type systems

— Python is not only dynamic typing

Until this moment, Python could be considered a programming language with a Dynamic and Structural type system, but what does this mean?

Exists many ways to define a type system, for example, you can define the system according to when the type verification is done, in this case, exists two main categories:

1. Dynamic typing. The type verification is realized at runtime. You need to perform the execution of the instruction to make the verification and this is the default behavior of Python.
2. Static typing. The type verification is realized at compile time with the help of the types annotated in the source code. In this case, is not necessary to execute the program to make the verification, the type checker makes a verification according to the static state of the program (source code).

In other cases, you can define the type system according to what checks the type system:

1. Nominal. Is the type system supported by traditional languages like Java, C++, etc and makes the validation according to the type or class of the object, example: You can only pass an object of type Person or some subclass (with polymorphism) if the function expects an argument of type Person .
2. Structural. Make the validation according to the methods that have the object, not the type. If the object implements all the required methods, then is a valid object. Note that is not required to make an explicit implementation of some interface, the only requirement is the implementation of the methods.

Python originally use the approach of Dynamic typing and motivates the use of Structural typing, but with the introduction of Type hints and Protocolos, now is possible to use Structural + Static typing.

Soon we will see what is a Protocol and how it is possible to generate this new hybrid type system.

Static Type Checkers

—When and how to validate the types in my code?

Now, you know that Python languages allow you to make static analyses of your code, but the Python interpreter does not validate the type annotations when you run the program. In this case, you need an additional program to analyze the type annotations and show the errors that exist in your code.

The program that checks the type annotations is known as a static type checker. This program implements a set of standard rules defined by the Python community that analyze your source code to give some feedback about some type errors in your source code.

Note: Note that the type checkers are static, this means that their analysis does not involve the execution of the source code, they only analyze the static representation of the program, in other words, the source code.

Some examples of static type checkers are:

• Pytype by Google.
• Pyright by Microsoft.
• Pyre by Facebook.
• Mypy.

In this series of posts, I going to use the Mypy type checker because is the most well-known checker in the ecosystem, and it was the project that introduced types in the Python ecosystem.

Some important points are as follows:

• When you execute your program with the Python interpreter, the type annotations are ignored. To verify the type annotations, you need to execute the type checker with the source code, for example: mypy my_program.py
• By default, the type checker should not emit warnings for blocks of code that do have type hints. If some variable is not annotated and does not have an assigned value in its declaration, then the type of the variable is typing.Any .
• Type hints do not provide some kind of performance advantage, but in theory, is possible to implement this kind of improvement in the future.

Generally, type checkers are not called manually, they are usually called automatically by the IDE when saving the file or in the CI pipeline.

Expect the second part very soon! In this new episode, we will see how type hints have evolved in Python, how new tools have been incorporated through the versions, and how you can implement these new features without losing expressiveness in your programs.

Conclusions

• Prefer Duck Typing.
• Duck typing is about capabilities rather than contracts.
• Type annotations in Python, allow you to generate a structural + static analysis.
• Exists many type checkers, for example, Mypy, PyRight, etc.

References

• Duck typing — Python Glossary
• Duck typing — Explanation
• Luciano Ramalho / Protocol: the keystone of type hints

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Join our community Discord.