🎡 cibuildwheel 2.19

 Posted on June 10, 2024  (Last modified on June 13, 2024)  |  Henry Schreiner

cibuildwheel 2.19 is out, with some very big additions. A new platform, Pyodide, has been added for building WebAssembly wheels. We’ve added CPython 3.13 free-threaded builds, now on all OS’s. And we have an opt-in speed improvement with the build[uv] build-frontend option.

We’ve had some fantastic releases of cibuildwheel since my last post over 2.10, so I’ll include a few of the new features from those releases, too, with a highlight on a larger feature that can use more explaining: inherit for overrides.

Pyodide wheels (2.19)

Cibuildwheel now supports building Pyodide wheels. This was around a year in the making; the PR started around PyCon last year and was completed around PyCon this year! This goes hand-in-hand with the new Pyodide 0.26.1 containing CPython 3.12. You can’t upload these wheels to PyPI (yet?), but you can use them in your websites using Pyodide, perfect for live documentation, for example.

To use, just build like this:

jobs:
  build-pyodide:
    name: Pyodide cibuildwheel
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: pypa/cibuildwheel@v2.19
        env:
          CIBW_PLATFORM: pyodide

      - ses: actions/upload-artifact@v4
        with:
          name: pyodide-wheel
          path: wheelhouse/*.whl

(cibuildwheel 2.19.1 fixed an issue that previously required a setup-python usage first.)

Some projects may require some extra flags. For example, I need the following flags for a pybind11 scikit-build-core project to enable exceptions and to tell pyodide that pybind11 correctly exports what it needs:

[[tool.cibuildwheel.overrides]]
select = "*pyodide*"
inherit.environment = "append"
environment.CFLAGS = "-fexceptions"
environment.LDFLAGS = "-fexceptions"
build-frontend = {name = "build", args = ["--exports", "whole_archive"]}

Pyodide-build is going to support its own configuration section which could be used instead, but I’m still having trouble using it in 0.26.1, so it might take a little time to be ready.

Building for free-threaded Python (2.19)

CPython 3.13 has two builds to target (the first time since 2.7, which had wide and narrow unicode builds). You have to acknowledge free-threading support in cibuildwheel currently with:

[tool.cibuildwheel]
free-threaded-support = true

Also remember CIBW_PRERELEASE_PYTHONS while CPython 3.13 is in beta. You can see an example using scikit-build-core (one of the first backends to support free-threading) in the samples repo. You can read more about free-threading and see the status of the ecosystem at Quansight-Labs/free-threaded-compatibility. Pybind11 support is in-progress.

Speeding up builds with uv (2.19)

uv is a Python package installer written in Rust that is 10-100x faster than pip in some cases. It’s also supported by pypa/build using the --installer=uv flag. We’ve added uv to the manylinux images, and now it’s an opt-in feature in cibuildwheel, as well. Here’s how you use it:

First, you need an external uv for Windows or macOS. If you are installing it yourself, you can request the [uv] extra, such as pipx run cibuildwheel[uv]. Here’s one easy way to get it in GitHub Actions:

- uses: yezz123/setup-uv@v4
- uses: pypa/cibuildwheel@v2.19

Then, you need to select the build[uv] backend:

[tool.cibuildwheel]
build-frontend = "build[uv]"

If you opt-in, the following will be true:

  • All installs and environment setup will be done with uv instead of virtualenv and pip
  • Build will be run with the --installer=uv option

A few things to keep in mind:

  • Extra packages like pip will not be installed; environments just contain what you ask for.
  • uv doesn’t support Python <3.8, so we’ll fall back to normal build there. Same is true for Pyodide.
  • There are two platforms uv doesn’t support: musllinux s390x (due to Rust limitations) and Windows ARM. There are no Windows ARM runners currently, though, and cross-compiling to Windows ARM from Intel (which is all cibuildwheel officially supports until there are runners) should be fine.
  • You must use a supported version of manylinux, either manylinux2014 (the default) or manylinux_2_28 (manylinux1, manylinux2010, and manylinux_2_24 were all discontinued some time ago).
  • uv handles pre-release dependencies a bit differently.

Remember, you can use overrides if you need to apply settings to subsets of selectors.

You might get best results by running multiple identifiers in one run, so that uv can take advantage of it’s excellent high-speed caching. This will (obviously?) not speed up the binary build process, but it will greatly reduce the overhead of running cibuildwheel as we set up environments and install packages a lot.

Inheriting (2.17)

A recent change is the ability to inherit and extend an environment. This looks like this:

[tool.cibuildwheel]
environment.FLAG = 1

[[tool.cibuildwheel.overrides]]
select = "*pyodide*"
inherit.environment = "append"
environment.CFLAGS = "-fexceptions"

This will combine the environment dict and both FLAG and CFLAGS will be passed to pyodide builds. Three settings are available: none (the default), append, and prepend. If you append, dict keys in the override take precedence, and lists are appended to.

Currently (up till at least 2.19), this doesn’t work correctly for config-settings. We will address this in a future update.

Smaller features

  • The container-engine option is no longer global (2.18.1)
  • CPython 3.13 betas (2.18)
  • MUSL defaults to musllinux_1_2 (2.18)
  • Support for native ARM runners like macos-14 on GHA (2.17)
  • --platform no longer required for local runs (2.17)
  • Added JSON Schema (also to SchemaStore) (2.16.2)
  • Extra flags supported for build-frontend (2.16)
  • SOURCE_DATE_EPOCH always passed through if present (2.16)
  • PyPy Apple Silicon support (2.12)
  • Cross-compiling Windows ARM wheels (2.11)

This is just a summary, lots of bug fixes and polish is not included above! We also now have a nice cibuildwheel.pypa.io docs site address.

Schema

You can check your pyproject.toml, including the latest cibuildwheel changes, with the following pre-commit-config.yml:

repos:
  - repo: https://github.com/henryiii/validate-pyproject-schema-store
    rev: 2024.06.10
    hooks:
      - id: validate-pyproject

Your favorite editors should also support this via SchemaStore; make sure you have the relevant TOML plugin enabled.

Update today!

If you are using dependenabot, you’ll need to update to v2.19, as adding the macOS free-threading could add wheels to an existing config, which causes us to require a minor version bump.

programming  python  cibuildwheel