I am pleased to announce the release of pybind11 2.6.0! This is the largest release since 2.2 (released over three years ago). I would like to highlight some of the key changes below; be sure to check out the changelog and upgrade guide for more information! The focus of this release was stability, packaging, and supporting more platforms, though there are a lot of small features and useful additions, covered by newly expanded docs.
Stability
One of the headline features is a brand new CI system, testing over 50 different
configurations. This has enabled expanded support for compilers, such as NVCC,
PGI, and lots of versions of Clang, GCC, and MSVC, now tested from C++11 to (the
newly supported) C++20. Many warnings and bugs were fixed across the spectrum.
PyPy support has been updated to the 7.3.x series, and fully supports pypy3
for the first time. New Python 3.9 support includes a workaround for a segfault
in CPython 3.9.0 (pybind11 developers have fixed the bug in CPython, and 3.9.1
will work with older versions of pybind11 too; conda-forge also provides the
patch). every configuration supported by cibuildwheel is supported.
CMake build system
Another headline feature is the completely reworked build systems; first we will cover CMake. The focus was to support more situations and modern CMake tools in a backward compatible way, while maintaining support for CMake 3.4+. Most significantly, the generated Config/Target files now no longer depend on the generator platform, allowing a configuration to be used on a different machine. This enables, for the first time, the CMake files to be distributed in the Python package available on PyPI via pip!
You now simply use CMAKE_CXX_STANDARD or compile features - pybind11 will
automatically require at least C++11 via compile features. You can use
CMAKE_INTERPROCEDURAL_OPTIMIZATION and CMAKE_CXX_VISIBILITY_PRESET and
pybind11 will get out of the way and let CMake dictate those features. CUDA as
a language is fully supported now. And the Python discovery is much smarter,
finding virtual environments and conda.
A huge new feature is support for CMake’s modern FindPython tooling; you can now simply write:
find_package(Python COMPONENTS Interpreter Development)
add_subdirectory(extern/pybind11)
and pybind11 will respect the Python you found, and will use the new variables.
You can force the new search by instead setting PYBIND11_FINDPYTHON ON before
including pybind11. This does requires CMake 3.12, with 3.15+ better and 3.18.2+
best, especially for PyPy (you also should use PyPy 7.3.2+).
You can now disable Python discovery entirely by setting
PYBIND11_NOPYTHON TRUE, and then do it yourself with the modular targets
provided. This is ideal for working in a large existing framework, or using
Scikit-build’s helper file.
There is much more to explore, as well, like finer-grained targets and new helper functions.
Setup helpers
Not to be outdone, the setuptools packaging has been completely reworked and
expanded to better support both simple and advanced setuptools projects. Before
2.6.0, pybind11 extensions that wanted to be widely available and avoid
requiring CMake had to follow the python example repo. The basic ideas in
that repo have now been integrated into a pybind11.setup_helpers module, and
can be used any way that you would like to use pybind11. A lot of long-standing
issues with the example have been fixed; if you have ever seen a setup.py with
the (incorrect) statement that “has_flag” is part of Python 3.6+, you know
that someone was copying this example. You can also be pretty sure that they
have an incorrect has_flag function, as well, that will generate temporary
files in your working directory on Windows. These and other bugs have been fixed
for setup_helpers; it properly supports Windows compilers, C++17, etc.
Using it is a breeze. Just use PEP 518’s pyproject.toml file to list pybind11,
or if you use the submodule form instead, just do:
DIR = os.path.abspath(os.path.dirname(__file__))
sys.path.append(os.path.join(DIR, "extern", "pybind11"))
from pybind11.setup_helpers import Pybind11Extension # noqa: E402
(better yet, use pathlib if you are Python 3 only). Building an extension is simple:
ext_modules = [
Pybind11Extension(
"myext",
sorted(SRC_FILES),
include_dirs=INCLUDE_DIRS,
cxx_std=11,
extra_compile_args=["/d2FH4-"] if sys.platform.startswith("win32") else [],
)
]
This looks just like a standard Extension (and is a subclass of it), but will
set up the flags needed to compile pybind11 extensions. It also has a cxx_std
setting that selects a C++ standard to use. There is also a pybind11_include
argument that can disable the inclusion of the pybind11 directory (the
setup_helpers file is completely standalone, and can even be copied into your
project or even your setup.py).
Note that above I added a normal addition for an extension; the
extra_compile_args. All normal additions are supported. (If you are curious,
the one I illustrate above will keep extensions that are built on MSVC 2019 from
adding an extra runtime dependency that some older redistributable downloads do
not have).
If you want to leave off the cxx_std and instead use the highest available
standard, then you need to also add the build_ext from setup_helpers
(currently, that is all this does, though in the future more functionality could
be added, like MinGW support):
from pybind11.setup_helpers import build_ext
setup(
build_cmd={"build_ext": build_ext},
)
There is also a parallel compiler utility, which can dramatically speed up the compile performance of your extension on multicore hardware:
from pybind11.setup_helpers import ParallelCompile
ParallelCompile("NPY_NUM_BUILD_JOBS").install()
The variable name you choose when constructing it allows users to control the number of threads (the one here matches NumPy, which has similar functionality). You can also control the default and max number of threads via keyword arguments.
New features
Besides stability and bugfixes, and major building improvements, there are a significant collection of exciting new features.
Keyword-only and positional-only arguments are now supported on any version of Python. Use them like this:
.def("my_func", &func, "p1"_a, py::pos_only(), "n1"_a, py::kw_only(), "k1"_a)
This will define a function with one positional only argument, a normal argument
n1, and a keyword-only argument k1. This can help you design tight, clean
interfaces that are easy to maintain. A bug with .none() has also been fixed
here.
You can now use py::is_final() to force a class to be uninheritable (CPython
only). You can use py::prepend() to insert a function at the top of the call
stack instead of last. There’s also now a way to get the type object from a
registered C++ type with py::type::of<T>(), or you can get the type from a
Python object with py::type::of(ob).
Support for code correctness
In several ways, pybind11 now helps you write more correct code. If you forget
to call __init__ on a pybind11 class, an error is thrown (since the C++ memory
is not initialized correctly without it). Conversions to python types, like
py::bytes from general types like py::object now throw if the conversion is
invalid, rather than having to check py::isinstance first.
Minor features
Lots of small changes were made to better support C++20 or be clearer/cleaner. There no longer is a nested submodule in the repo (great for use as a submodule from other places, like GitLab). Check out the full list, there are bugs finally fixed after several years in some cases! The docs have also been expanded and feature several new pages and sections.
New maintainer team
The largest change might not be in the code at all. A new collection of some of the best contributors were pulled in and made part of the maintenance team. There are more than a half-dozen of us, and we cover a wide range of backgrounds and interests, from High Energy Physics to Google. We would also like to thank Google for a generous donation to help our large number of CI jobs to run fully in parallel.
There are more exciting improvements to come! Hopefully releases will be a little more regular in the future with more patch releases and more frequent releases. Please continue to help us catch issues early by testing with the master branch! We can sometimes pull or change a feature on the master branch, but it should be quite stable, as it has to pass our 50+ checks.