Programming

C++23 is complete and pretty much finalized, so let’s explore the new features in C++, from a data science point of view. This is not as large of a release as C++20 or C++11, but it’s still a hefty release with lots of nice new features - larger than C++17.

Some of the major new features are building on themes from C++20, such as vastly expanded ranges and views, more constexpr/consteval, module support for the standard library, std::print to supplement std::format, and a std::generator to support coroutines. There are also several general features, lambdas and classes get along much better now, there are new preprocessor additions (!), a stacktrace library, flat maps/sets, and new std::optional monadics, deducing this, and many various additions.

One outcome of the 2023 Scientific Python Developer Summit was the Scientific Python Development Guide, a comprehensive guide to modern Python package development, complete with a new project template supporting 12 build backends and a WebAssembly-powered checker with checks linked to the guide. The guide covers topics like modern, compiled, and classic packaging, style checks, type checking, docs, task runners, CI, tests, and much more! There also are sections of tutorials, principles, and some common patterns.

This guide (along with cookie & repo-review) started in Scikit-HEP in 2020. During the summit, it was merged with the NSLS-II guidelines, which provided the basis for the principles section. I’d like to thank and acknowledge Dan Allan and Gregory Lee for working tirelessly during the summit to rework, rewrite, merge, and fix the guide, including writing most of the tutorials pages and first patterns page, and rewriting the environment page as a tutorial.

I’ve released a new¹ toolkit for running checks, similar to Ruff and Flake8 but designed to check configuration, called repo-review. It requires Python 3.10+² to run and has no built-in checks, but is easy to write plugins for. A set of checks based on the Scientific Python Development Guide (which I also have a post about!) are available as a plugin, sp-repo-review. You can run repo-review in WebAssembly (via Pyodide), or in pre-commit, or as a GitHub Action. It supports multiple output formats, including Rich, HTML, and JSON. The system is based on fixtures (like pytest) and topologically sorts requirements. You don’t need to depend on repo-review to add a repo-review plugin. You can see a live version using sp-repo-review in-place here or standalone here.

Python 3.12’s beta’s are out, which means the features are locked in. The theme this year has been cleanup and typing. distutils has been removed, and setuptools is no longer present in default environments.

cibuildwheel 2.10 is out, with some important additions. PEP 517 config settings added, --only (which has an interesting use in GHA), and Cirrus CI support (including our first Apple Silicon native runner!) are highlights. We also support Python 3.11 now (as of 2.11.2, RC’s in older releases).

We’ve had some fantastic releases of cibuildwheel this year, including some very powerful features you might be interested in using, and I haven’t covered releases since 2.2, so let’s take an in-depth look at what’s new for this and the last few releases!

Python 3.10 is out (and has been for a while, I’m late posting this), with new features and changes. The big new language feature this update is pattern matching. We get much better errors, the always-present typing improvements, and finally some real usage of the new PEG parser from 3.9.