Python 3.7

Python 3.7 has been out for a while. In fact, it’s the oldest version of Python still receiving support when this was written. I’d still like to write a “what’s new”, targeting users who are upgrading to a Python 3.7+ only codebase, and want to know what to take advantage of!

Future annotations

If you add the following line to a Python module:

from __future__ import annotations

all of your annotations will remain unevaluated strings. This means you can use the latest Python typing syntax in your code (which is much more readable) and still support Python 3.7+! There are two downsides: you can’t use the new syntax outside of annotations (obviously), and you can’t access the annotations at runtime. So this import is highly recommended unless you are using something using runtime annotations (like cattrs, pydantic, typer, or even functools.singledispatch).

Module access

This is something you could already be using, but now it can be included in your design. Two new module level functions were added: __dir__() -> List[str] and __getattr__(name: str) -> ANy. Most modules that might be imported in a REPL should already include this function:

def dir() -> List[str]:
    return __all__

This will allow tab completion to skip anything not in your __all__, like the annotations that you imported from __future__ along with all of your other imports.

The __getattr__ is really exciting, because it enables you to do all sorts of dynamic attributes on modules. You could make from plubmum.cmd import <any shell command> work in much less code. You could catch a common misspelling (like hist.axes instead of hist.axis), print a warning, and then make it work anyway. I would probably not mix with difflib to produce correction suggestions, because Python 3.10 does this for you anyway, and REPL users should upgrade. But you could (just limit it to <3.10, using the standard feature is better there).

Core support for typing on classes

Two new special methods were added. __class_getitem__ allows SomeClass[thing] to be supported. __mro_entries__ is called during subclassing to get the bases if you subclass something that is not a class; if it is called, __orig_bases__ contains the original bases, and __bases__ contains the thing this triggers. It is used because List[int] does not actually return a class, but an instance - that instance knows how to get the proper bases (list, object) because that’s what mro_entries returns. So you can now use instances instead of classes when subclassing, and those instances can replace themselves with classes during the process.

Other changes

• async and await are now keywords, but hopefully you weren’t using them. I wonder if they could go back to being soft keywords in the future, with the new parser and soft keyword specification?
• breakpoint() is great for debugging, and has a hook for IDEs and such to support it.
• The built-in dict is now required to be ordered, though it was also ordered in CPython 3.6 and PyPy, so I’d be tempted to call that a 3.6+ feature.
• New thread-local storage C-API, if you are writing extensions.
• Importing typing is now 7x faster.

Other standard library things

• Dataclasses is now available in the standard library. But since there’s a 3.6 backport, I will assume you were able to start using them a version ago.
• contextvars was added, if you need those.
• importlib.resources was added, but again, there’s a backport, and the backport has a nicer API from the 3.9+ stdlib that I’d use instead.
• Major updates to asyncio; while still provisional, it’s close enough to being usable the way it ended up being set.
• Nanosecond resolution time functions (can be 3x better than time.time()).
• nullcontext & async context managers in contextlib.
• functools.singledispatch supports type annotations.
• subprocess.run now has capture_output=, text=, and handles KeyboardInterupt a little better.

Other developer changes

Library developers may need to be aware of the following changes:

• C-API for thread local storage
• Deprecation warnings improvements, formalizing PendingDeprecationWarning -> DeprecationWarning -> FutureWarning.
• Hash-based .pyc files.
• -X dev, Python development mode (slower but safer).
• -X importtime to measure time taking doing imports
• More than 255 arguments on a function allowed. Who discoverd this, and what were they doing…

Final words

This was a fantastic release - and is the last “large” (18 month) release of Python ever. Python moved to a 12 month release cycle after 3.7. As of 2022, this is the oldest officially supported version of Python, and already has been dropped by the data science libraries following NEP 29 / SPEC 0.

Sources

This blog post was written well after the release, so the primary source was the docs & experience. Here are a few possibly useful links:

• Official docs
• RealPython article

Bonus: Automatic upgrades

I will assume you have already run pyupgrade with --py37plus, ideally added it to pre-commit. This will clean up a lot of things automatically. Unique to the 3.7+ upgrade is from future import annotations; adding this to all your files will enable pyupgrade to also clean up your type annotations. You can use isort to inject this automatically. All told, this is what your pre-commit file should look like:

- repo: https://github.com/PyCQA/isort
  rev: "5.10.1"
  hooks:
  - id: isort
     args: ["-a", "from __future__ import annotations"]

- repo: https://github.com/asottile/pyupgrade
  rev: "v2.31.0"
  hooks:
  - id: pyupgrade
    args: ["--py37-plus"]

This will at least fix:

• Type annotations are strings (can use any syntax supported by mypy)
• subprocess.run changes universal_newlines= into text=

More might be added in future versions.