In this post, we will be taking on Python type hinting and discuss the subtle differences between object and Any types and how static type checkers behave based on our instructions.
There are a few static type checkers for Python like mypy, pytype and pyright to name a few, that help bring the assurance of static typing over to Python while retaining the convenience of dynamic typing. We picked mypy to illustrate our examples here.
Generally, we see the Any type used pretty often, even when object would be better. In this post, we are going to focus on the difference between object and Any when utilizing Python type hinting and static type checking.
A primer on type hinting and static type checkers
At its core, Python is a dynamically or duck typed language. But as your projects grow and more people start working on the same project, static typing becomes more attractive.
Starting from Python 3.5, we have the option to statically indicate the type of a value in our Python code. Or in other words, type hint it.
Let’s see how we can type hint a variable that holds an integer and then assign the value 42 to it:
>>> my_integer: int
>>> my_integer = 42 # we can also do it in one line:
>>> my_integer: int = 42
From now on, whoever sees that variable, either by first locating its declaration or by hovering her mouse on it in the IDE, knows it is supposed to be an integer.
Nevertheless, type hinting on its own will not enforce any integrity. In the example above, we could assign a string to the my_integer variable and would not face any errors. That’s why we require some additional help from static type checkers.
Static type checkers like mypy leverage type hinting and take on the responsibility of evaluating our code and making sure the assigned values to variables are what we intended them to be.
Benefits of incorporating type hinting and static type checkers
Below are some of the advantages that static typing brings to the table:
- Improves readability by making function/method signatures more descriptive, so you can often see a method’s intent at a glance.
- Keeps your codebase clean and less prone to having bugs.
- Reduces human error
- Supercharges your IDE
Let’s see how much of a difference it makes when we use one type over the other.
object vs Any
Since type hints and static type checking are additions to Python, it is common for projects to adopt them gradually. At Inmanta, we try to maximize type coverage but there are still parts of the codebase that are untyped. This gradual adoption is where the Any type comes in.
The purpose of the Any type is to indicate to the type checker that a part of the program should not be checked. A variable (or function parameter) that is annotated with the Any type accepts any value, and the type checker allows any operation on it.
On the other way around, a variable of any type accepts an object of type Any. This means the following snippet will type check correctly, even though it will not run correctly:
from typing import Any
my_int: Any = "This is a string!"
my_int.my_method()
my_bool: bool = my_int
Annotating parts of your program with Any can be useful to have static checking without getting a whole bunch of type errors for the sections that haven’t been annotated yet. The semantics of the Any type outlined above make sure the type checker never raises an error for an Any typed variable/value.
Pitfall
However, there is a pitfall: due to its name, we have found that the semantics are often misinterpreted. Aside from being used with the intent to blind the type checker to a part of the program, it is often used when the programmer wants to indicate that a variable/parameter accepts any object.
While this may sound similar at first sight, there are some very important differences. When the Any type is used for situations like this, even though it wasn’t the intention, it still indicates to the type checker not to bother with that part of the code involving that variable/parameter.
Below, we will supply an example scenario, illustrating the issues that might arise when using Any and propose an alternative that matches closely with what the programmer intends.
Consider a simple (and rather redundant) print function:
def myprint(obj):
print(obj)
The function is meant to handle any value we pass to it. A naive approach to adding type annotations would then be:
def myprint(obj: Any) -> None:
print(obj)
At the moment the consequences of this misinterpretation of the semantics of Any are limited. But now suppose we decide to expand on the behavior of our print function. We don’t want it to just print the object, but to say hello to it as well.
Easy! A simple string concatenation will do:
def myprint(obj: Any) -> None:
print("hello " + obj)
myprint("world!")
myprint(42)
Let’s type check it:
/home/Python$ mypy main.py
Success: no issues found in 1 source file
The type checker reports no errors. Great! Let’s give it a go:
/home/Python$ python main.py
"hello world!"
Traceback (most recent call last):
File "/home/Python/main.py", line 9, in <module>
myprint(42)
File "/home/Python/main.py", line 5, in myprint
print("hello " + obj)
TypeError: can only concatenate str (not "int") to str
Why didn’t the type checker warn us about this? Because we inadvertently told it not to!
The obj parameter is of type Any, so the type checker allows us to do anything we feel like with it, for instance using the + operator on it. This was not the intention when adding type annotations; the goal is to have the type checker catch mistakes like this, not hide them behind an Any type.
The message we want to send to the type checker is that obj might be a value of any type and we don’t know which, so please don’t make any assumptions. Instead, we’ve sent: it can be of any type, please assume anything I do with it is valid, because it could be.
object to the rescue
So, we have determined that the Any type is not appropriate in this case. The object type on the other hand is exactly what we need. It is the supertype of all types, so if we write:
def myprint(obj: object) -> None:
print("hello " + obj)
myprint("world!")
myprint(42)
And run the type checker:
/home/Python$ mypy main.py
main.py:4: error: Unsupported operand types for + ("str" and "object") Found 1 error in 1 file (checked 1 source file)
The type checker is still satisfied with us passing it “world!” and 42 because both str and int are subtypes of object. Now, contrarily, it tells us Unsupported operand types for + (“str” and “object”), discovering the mistake we accidentally put in our program.
Let’s backtrack a bit and have another look at a snippet we looked at earlier, this time using object:
/home/Python$ cat main.py
my_int: object = "This is a string!"
my_int.my_method()
my_bool: bool = my_int
Running mypy yields:
/home/Python$ mypy main.py
main.py:2: error: "object" has no attribute "my_method"
main.py:3: error: Incompatible types in assignment (expression has type "object", variable has type "bool")
Found 2 errors in 1 file (checked 1 source file)
The first line still type checks correctly: a string is an object so that assignment is fine.
The next two lines give type errors: my_int can be any object, and not all objects have a my_method method, nor are all objects booleans.
The same is true for function return values. If a function is typed as returning Any, the type checker will not make any checks on what the caller does with it.
How does it affect us?
At Inmanta we place a great emphasis on type hinting our codebase. We closely monitor the coverage of our codebase and constantly improve it. Also, we use a handful of open-source tools, and whenever possible we contribute back to the community.
During one of our sessions revolving around the type hinting of the Python logging module in Typeshed, we observed the use of type Any in places where utilizing the object type would be more appropriate. It motivated us to make a contribution to the Typeshed project to improve the typing of the logging module.
The improvement of type coverage introduced by this change is evident in the graph below:
If you want to learn more about this change, you can find our pull request here.
Summary
This post illustrated the nuances of Python type hinting. The Any type is very useful for gradually moving from an untyped codebase to one with type annotations used for static type checking. It hides untyped parts of the program from the type checker. In practice, however, it’s often (mis)used in situations where the intention is to state that a variable should accept any type of value. This has the side effect of hiding that part of the program for the type checker as well. It results in a weaker guarantee provided by the type checker.
The solution is to use object instead. Rather than hiding a part of the program from the type checker, it signals that a variable can be of any type. So anything we do with it, has to be sufficiently generic that we can apply to all types.
In order to read further on these subjects and to also get familiar with the concept of gradual typing, as well as type hinting, we recommend taking a look at PEP483 and PEP484.
Special thanks to Sander Van Balen and Wouter De Borger for co-authoring this post.