cmake

Scikit-build-core 0.10 is out, and it is one of the largest releases we’ve produced in terms of new configuration options. It automatically reads your minimum CMake version, you can tell it to read your scikit-build-core minimum-version, and there are settings for many of the advanced things users used to do with scikit-build (classic), like rebuilds or pure Python fallbacks on build failures.

I’ve spent the last few years trying to make it easy for anyone to extend Python with compiled languages. I’ve worked on pybind11, a powerful C++ library that allows users to write advanced Python extensions using just C++11, used by some of the largest projects, SciPy, PyTorch, Google, LLVM, and tens of thousands of other libraries, down to very small extensions. I also work on cibuildwheel, which makes building binaries (called wheels) on continuous integration (CI) simple. It is again powerful enough to used by huge projects, like Scikit-learn, matplotlib, mypy; and is simple enough to be used by hundreds of other packages. Recently it was accepted into the Python Packaging Authority (PyPA). There is one missing piece, though, to complete this picture of compiled extensions that easy to use for small projects, and powerful enough for large projects: the build system. I believe the solution to that is scikit-build, and I’d like to work on it over the next three years.

Scikit-build is a tool for integrating a package with a CMake build system into Python. You can utilize the vast collection of packages and projects using CMake already, and you have access to modern building features, like multithreaded builds, library discovery, superb compiler and IDE support, and all sorts of extended tooling. Modern CMake is quite pleasant to write compared to times past; I have written a book and training course on it. We ship up-to-date cmake and ninja wheels for all binary platforms.

Update: Funded! I’ll be working on this starting August 1, 2022!

I wrote a proposal for an NSF CSSI Elements project containing three parts. The first part will cover core development on Scikit-build to address the current shortcomings and to prepare it for a post-distutils (Python 3.12+) world. The second part would cover assisting libraries with a science use case in either transitioning to scikit-build (ideally from an existing CMake build system with Python bindings, but I can help mentor developers in writing bindings (ideally pybind11), setting up CI, and writing CMake code as well (see my book or workshop on Modern CMake, and I’m happy to help old scikit-build projects transition to better practices). As part of this, I would be building up the examples and documentation, leading into the third part of the proposal: A series of training events and training material, including plans for something alongside SciPy.

You can also see an outline at scikit-build/scikit-build/wiki or at the end of this post.

Thank you for all the projects! The proposal was submitted Dec 8, 2021; mid year we should find out if it was accepted!

CMake 3.11 was just released; this is particularly exciting release for CMake. I’d like to give a quick and friendly introduction to the new features that might make the largest difference for CMake users.

GooFit 2.1 introduces the full-featured Python bindings to GooFit. These bindings mimic the C++ usage of GooFit, including bindings for all PDFs, and also provide NumPy-centric conversions, live Jupyter notebook printing, pip install, and more. Most of the examples in C++ are provided in Python form, as well.

Several other API changes were made. Observables are now distinguished from Variables and provided as a separate class. Both these classes are now passed around by copy everywhere.¹ The three and four body amplitude classes have been refactored and simplified. OpenMP is now supported via homebrew on macOS; GooFit is one of the only packages that currently can build with OpenMP on the default macOS compiler. Eigen is now available, and CLI11 has been updated to version 1.3.

GooFit 2.1 will receive continuing support while development on GooFit 2.2 presses on with a new indexing scheme for PDFs.

The next version of the premier CUDA/OpenMP fitting program for HEP analysis, GooFit 2.0, has been released. GooFit is now easy to build on a wide variety of Unix systems, and supports debuggers and IDEs. GooFit is faster, has unit tests, and working examples. More PDFs and examples have been added, as well as newly released example datasets that are downloaded automatically. GooFit now has built in support for MPI, and can use that to deploy to multiple graphics cards on the same machine. A new command line parser (CLI11) and drastically improved logging and errors have made code easier to write and debug. Usage of GooFit specific terminology is now reduced, using standard Thrust or CUDA terms when possible, lowering the barrier for new developers. A new Python script has been added to assist users converting from pre 2.0 code.

The file structure of GooFit and the build system have been completely revamped. The fake nvcc features have been removed, as have the rootstuff copies of ROOT classes. PDFs are now organized by type and compile and link separately. Multiple PDF caching support has improved. The build system now uses CMake and manages external libraries.

A new feature of the CMake build system is GooFit Packages, which are complete packages that can be added to GooFit and built, allowing analysis code to live in a separate location from GooFit, rather than the old method of simply forking GooFit and adding your analysis manually. A GooFit Package can be made into an example trivially. See this package for an example.

GooFit 2.0 will receive continuing support while development on GooFit 2.1 presses on.

Links:

GooFit on GitHub • GooFit webpage • API documentation

This is a quick recipe for setting up CMake to use googletest in your projects. First, make a tests folder in the root of your project. Then, add add_subdirectory(tests) to your CMakeLists.txt, after you’ve finished adding the libraries in your project. Note that the way I’ve written this probably requires CMake 3.4+.