Rad's blog

Programming, Photography, Guides, Thoughts

Python 3.9 EOL, Generated by OpenAI DALL·E 3

Python 3.9 End of Life

—

by

Radek Lát
in

Python 3.9 has rea­ched its end of life 26 days ago. However, this time around, vari­ous tools and lib­ra­ries have alrea­dy star­ted remo­ving its sup­port. If you have the opti­on to upgra­de to the latest Python ver­si­on, take it. But espe­ci­ally lib­ra­ry main­ta­i­ners don’t have that luxu­ry and still have to keep 3.10 sup­port around. It’s been a whi­le sin­ce all the „What’s new in Python 3.10” articles came out. So let’s recap what you can refactor.

Befo­re we go into the details of what’s chan­ging, I would recom­mend try­ing the pyupgrade (UP) rules of ruff first to see what Python 3.10+ chan­ges can be done for you auto­ma­ti­cally. For cases whe­re ruff is not suf­fi­ci­ent, you can use AI assistan­ce with a mas­ter prompt at the end of this article.

Type Unions, Aliases, and Guards

Python 3.10 intro­du­ces a new uni­on type syn­tax that’s more rea­da­ble and con­ci­se. You can now refac­tor your type hints to use the cle­a­ner | operator:

from typing import Union, Optional, List, Dict

def process_data(items: List[Union[str, int]]) -> Optional[Dict[str, Union[str, int]]]:
    if not items:
        return None
    return {"count": len(items), "first": items[0]}
def process_data(items: list[str | int]) -> dict[str, str | int] | None:
    if not items:
        return None
    return {"count": len(items), "first": items[0]}

You can also use the new TypeAlias for bet­ter type documentation:

UserId = int

def get_user(user_id: UserId) -> None:
    ...
from typing import TypeAlias

UserId: TypeAlias = int

def get_user(user_id: UserId) -> None:
    ...

ParamSpec and Concatenate

Ano­ther new typing fea­tu­re of Python 3.10 is a para­me­ter spe­ci­fi­cati­on vari­a­ble ParamSpec. It can be used in anno­tati­ons of Callable inste­ad of .... Exis­ting TypeVar can­not be used here because it repre­sents a sin­gle vari­a­ble, whi­le the new ParamSpec repre­sents mul­tiple vari­a­bles. Addi­ti­o­nally, you can use Concatenate to add para­me­ters to this set.

# Before ParamSpec, you could only use `...`, which is not type-checkable

from collections.abc import Callable
from threading import Lock
from typing import TypeVar

R = TypeVar('R')

def with_lock(f: Callable[..., R]) -> Callable[..., R]:
    ...
# With ParamSpec, you can annotate the parameters as well
# The return value drops the `Lock` parameter and keeps the rest.

from collections.abc import Callable
from threading import Lock
from typing import Concatenate, ParamSpec, TypeVar

P = ParamSpec('P')
R = TypeVar('R')

def with_lock(f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]:
    ...

Structural Pattern Matching (PEP 634)

Python 3.10 intro­du­ces structu­ral pat­tern matching, which allows you to refac­tor com­plex if-elif cha­ins into more rea­da­ble match statements:

def handle_response(response):
    if isinstance(response, dict):
        if response.get("status") == 200:
            data = response.get("data")
            return f"Success: {data}"
        elif response.get("status") == 404:
            return "Not found"
        elif response.get("status") == 500:
            error = response.get("error")
            return f"Server error: {error}"
        else:
            return "Unknown response"
    else:
        return "Invalid response"
def handle_response(response):
    match response:
        case {"status": 200, "data": data}:
            return f"Success: {data}"
        case {"status": 404}:
            return "Not found"
        case {"status": 500, "error": error}:
            return f"Server error: {error}"
        case _:
            return "Unknown response"

You can also refac­tor class-based conditionals:

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

def describe_point(point):
    if point.x == 0 and point.y == 0:
        return "Origin"
    elif point.x == 0:
        return f"On Y-axis at {point.y}"
    elif point.y == 0:
        return f"On X-axis at {point.x}"
    else:
        return f"Point at ({point.x}, {point.y})"
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

def describe_point(point):
    match point:
        case Point(0, 0):
            return "Origin"
        case Point(0, y):
            return f"On Y-axis at {y}"
        case Point(x, 0):
            return f"On X-axis at {x}"
        case Point(x, y):
            return f"Point at ({x}, {y})"

Stricter Zipping of Sequences

Python 3.10 / PEP-618 adds a strict flag to zip(). When ena­bled, zip will rai­se a ValueError when the zip­ped sequen­ces are not of equal len­gth. This can help catch bugs. If not sup­plied, the beha­vi­our is rema­ins the same – to igno­re the extra elements.

# Old way - this silently truncates, which might be a bug
print(list(zip(['a', 'b', 'c'], [1, 2])))
>>> [('a', 1), ('b', 2)]
# Same as in 3.9 but intentionally truncating
print(list(zip(['a', 'b', 'c'], [1, 2], strict=False)))
>>> [('a', 1), ('b', 2)]
# Raises ValueError, non-equal length explicitly forbidden
print(list(zip(['a', 'b', 'c'], [1, 2], strict=True)))
>>> ValueError

Context Manager Syntax

You can refac­tor nested con­text managers to use the cle­a­ner syntax:

def process_files():
    with open('file1.txt') as f1:
        with open('file2.txt') as f2:
            with open('output.txt', 'w') as out:
                # process files
                pass
def process_files():
    with (
        open('file1.txt') as f1,
        open('file2.txt') as f2,
        open('output.txt', 'w') as out,
    ):
        # process files
        pass

AI Prompt

You can try pas­sing the following AI prompt to an AI agent of your cho­ice for auto­ma­tic refac­to­ring that ruff may have mis­sed. Some refac­to­ring oppor­tu­ni­ties can be iden­ti­fied even in code bases that have been at Python 3.10+ for a long time.

You are a Python expert helping to refactor code to use Python 3.10+ features. Follow these specific refactoring patterns.

# Process
1. Check out the git trunk branch (master/main).
2. Pull the latest changes.
3. Switch to a new git branch for this refactoring.
4. Create a todo list with one item for each pattern, then tackle them one by one.
5. Separate each pattern changes to a new commit. Assume you are in the repository root and run just `git -a -m "<MESSAGE>"`.

## For each change
- Maintain the same functionality
- Ensure backward compatibility if needed

# Refactoring patterns

## Type Hints

Replace `typing.Union[X, Y]` with `X | Y`, `typing.Optional[X]` with `X | None`, and `typing.List/Dict/Tuple` with `list/dict/tuple` generics.

```
# OLD
from typing import Union, Optional, List, Dict, Tuple
def func(x: Union[int, str]) -> Optional[List[Dict[str, Union[str, int]]]]:

# NEW
def func(x: int | str) -> list[dict[str, str | int]] | None:
```

Also works for class and instance checks.

```
isinstance("mypy", str | int)
issubclass(str, int | float | bytes)
```

## ParamSpec and Concatenate

Use `typing.ParamSpec` to annotate `...` previously used inside `Callable`. Use `typing.Concatenate` if any parameters are dropped or added in the return value.

```
# OLD
from collections.abc import Callable
from typing import TypeVar

R = TypeVar('R')

def func(f: Callable[..., R]) -> Callable[..., R]:
    ...

# NEW
from collections.abc import Callable
from typing import Concatenate, ParamSpec, TypeVar

O = ParamSpec('O')
P = ParamSpec('P')
R = TypeVar('R')

def func(f: Callable[Concatenate[O, P], R]) -> Callable[P, R]:  # Result drops parameter `O`
    ...

def func(f: Callable[P, R]) -> Callable[Concatenate[O, P], R]:  # Result adds parameter `O`
    ...
```

## Structural Pattern Matching

Convert complex if-elif chains to match statements where appropriate, especially for data validation and handling different response types.

```
# OLD
if isinstance(data, dict):
    if data.get("status") == 200:
        return f"Success: {data.get('data')}"
    elif data.get("status") == 404:
        return "Not found"
    else:
        return "Unknown"

# NEW
match data:
    case {"status": 200, "data": data}:
        return f"Success: {data}"
    case {"status": 404}:
        return "Not found"
    case _:
        return "Unknown"
```

## Dictionary Operations

Replace `{**d1, **d2}` with `d1 | d2` and `d1.update(d2)` with `d1 |= d2`.

```
# OLD → NEW
result = {**dict1, **dict2}

# NEW
result = dict1 | dict2
```

```
# OLD
dict1.update(dict2)

# NEW
dict1 |= dict2
```

## Zip Strictness

Add a `strict` flag to `zip()` calls. Ask user if they prefer `strict=True` to ensure equal-length sequences, or `strict=False` for the old, truncating and backward compatible behavior.

```
# OLD
list(zip([1, 2, 3], ['a', 'b']))

# NEW
list(zip([1, 2, 3], ['a', 'b'], strict=False))  # for truncation and backward compability
list(zip([1, 2, 3], ['a', 'b'], strict=True))   # for error on mismatch
```

## Context Managers

Refactor nested with statements to use the cleaner multi-context manager syntax.

```
# OLD
with open('file1.txt') as f1:
    with open('file2.txt') as f2:
        with open('output.txt', 'w') as out:
            # process

# NEW
with (
    open('file1.txt') as f1,
    open('file2.txt') as f2,
    open('output.txt', 'w') as out,
):
    # process
```

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.