The cibuildwheel package has just had a major release with some fantastic
features. Python 2.7 and 3.5 support has been removed (and PyPy3.6), allowing us
to update to the latest manylinux and auditwheel versions, and support the newly
unified manylinux PyPy3.7 images. We now allow users to select pypa/build as a
build frontend. We now have a custom option to enable pre-release Pythons (3.10
currently) for testing before they are ABI stable (please don’t release wheels
until that happens). Maybe most exciting, cibuildwheel now supports
configuration in pyproject.toml, allowing you to be even further isolated from
dependence on your CI system; you can easily produce Linux and Windows wheels
locally (macOS still installs to system locations). And, since my
last post and
introduction post, cibuildwheel is now part
of the PyPA!
Config mode
The most exciting new feature is the new config mode. This allows you to specify
anything you would have added as environment variables in your pyproject.toml
file as static configuration. For example, almost every package activates
cibuildwheel’s fantastic testing system, which installs the newly build wheel in
a fresh environment and runs whatever command you give. Before, this meant that
each CI you used had to have an environment variable specification like this for
each CI provider:
env: CIBW_TEST_EXTRAS = "test" CIBW_TEST_COMMAND = "pytest {project}/tests"
Now, you can add the following to your pyproject.toml:
[tool.cibuildwheel]
test-extras = "test"
test-command = "pytest {project}/tests"
And, if you run this locally, you don’t have to add those environment variables manually!
CIBW_BUILD="cp39*" pipx run cibuildwheel --platform linux
You can override any setting with an environment variable, and you can specify a
different file name with --config-file if you want to.
So what should you specify in TOML, and what should remain environment
variables? I would recommend all static settings be in your pyproject.toml,
such that a default run of cibuildwheel does something sensible. Configuration
(including platform-specific configuration), skipping things you do not support,
etc. Variables that depend on which matrix item you are in, such as
CIBW_BUILD, should remain environment variables. Things that are specific to
the machine you are on (like Linux archs) likely should remain command line /
environment variables too. MacOS architectures probably could be specified in
TOML, however, since that simply requires a recent Xcode. For example:
[tool.cibuildwheel.macos]
archs = ["auto", "universal2"] # or whatever you support
test-skip = ["*universal2:arm64"]
The docs now contain tabs with both configuration methods for each setting, along with improved examples. cmake is a great example of a rather complex configuration setup. There is a separate documentation branch for the 1.x series if you need to access the information about working around Python 2 quirks.
Build selection
You can now select build frontends; the classic “pip” frontend, or pypa/build.
[tool.cibuildwheel]
build-frontend = "build"
Please try it out and report any problems you come across. Eventually, we might
change the default to build. You should already be building your SDists with
pipx run build --sdist.
Python support
We now support pre-release Pythons that are not yet recommended for wheel
upload. To enable this, set CIBW_PRERELEASE_PYTHONS or pass
--prerelease-pythons on the command line. It is not a good idea to “burn” this
into your pyproject.toml, as it’s only for testing. When Python enters RC
phase, you can start uploading extensions to PyPI, so the next version of Python
(3.10 currently) will leave the prerelease phase.
PyPy unification
We integrated PyPy3.7 into the official manylinux images, so besides the
manylinux2010-based image that was previously available, you can now build
with the official manylinux2010, manylinux2014 and manylinux_2_24 images
(regardless of if you use cibuildwheel or not). We also are using PyPy 3.7.5
now, which fixes several potential issues, and builds 64-bit wheels (only) on
Windows. This also means we support PyPy on ARM and 32-bit Linux. If you support
PyPy, it is highly recommended you upgrade to cibuildwheel 2.0.
Also, if you are using the manylinux images directly, you now can use
python3.Y shortcuts, and pipx is now installed; this means you can even use
manylinux to run tools like tox and nox! For example:
docker run --rm -itv $PWD:/src -w /src quay.io/pypa/manylinux_2_24_x86_64:latest pipx run nox
cibuildwheel uses this in its noxfile to run pip-tools compile on all
versions of Python, actually.
Everything else
We finally can use recent manylinux images (since manylinux dropped Python 2.7 and Python 3.5 some time ago) allow us to use auditwheel4, which produces slightly different wheel names - they include the classic tag as well as the new GLIBC based tag.
While it was backported to the 1.x series, support for Apple Silicon Python 3.8
wheels was added to the existing Python 3.9 support. The GitHub Action now
redirects stderr to stdout, fixing the out-of-order logs sometimes produces.
Developer tasks now use nox, simplifying our
maintainers and making it easier to contribute.
Wrapup
All told, there has never been a better time to build wheels for Python projects. The new configuration mode makes it easy to run Linux and Windows locally, and simplifies deployment on multiple CI providers. Python 3.10 support, PyPA/build, PyPy unification, and removal of old Pythons all make cibuildwheel more capable and powerful for building modern wheels.
We have plans for the future, too; better support for Limited API (stable ABI) projects (for testing mostly, they are easy to build today - just limit your build to one Python version). We hope to provide examples for extracting and building from an SDist. We hope to continue to improve the documentation, both ours,