Python

This is the first of two posts on cibuildwheel, a fantastic project I joined after switching to it from my own azure-wheel-helpers, which I’ve blogged about here before. It is the best wheelbuilding system available for Python today, and can make something that is normally a pain to setup and a headache to maintain a breeze (by forcing all the headaches on us, of course, as maintainers, but it’s better to solve issues centrally! Obviously we rather like solving these problems. Or we are just crazy, which is also possible ;) ).

Be sure to checkout the followup post over new features in 1.8.0 and 1.9.0, too! Also, cibuildwheel was recently accepted into the PyPA!

I am pleased to announce the release of pybind11 2.6.0! This is the largest release since 2.2 (released over three years ago). I would like to highlight some of the key changes below; be sure to check out the changelog and upgrade guide for more information! The focus of this release was stability, packaging, and supporting more platforms, though there are a lot of small features and useful additions, covered by newly expanded docs.

COVID-19 is ravaging the globe. Let’s look at the excellent Johns Hopkins dataset using Pandas. This will serve both as a guideline for getting the data and exploring on your own, as well as an example of Pandas multi-indexing in an easy to understand situation. I am currently involved in science-responds.

The foundational histogramming package for Python, boost-histogram, hit beta status with version 0.6! This is a major update to the new Boost.Histogram bindings. Since I have not written about boost-histogram yet here, I will introduce the library in its current state. Version 0.6.2 was based on the recently released Boost C++ Libraries 1.72 Histogram package. Feel free to visit the docs, or keep reading this post.

This Python library is part of a larger picture in the Scikit-HEP ecosystem of tools for Particle Physics and is funded by DIANA/HEP and IRIS-HEP. It is the core library for making and manipulating histograms. Other packages are under development to provide a complete set of tools to work with and visualize histograms. The Aghast package is designed to convert between popular histogram formats, and the Hist package will be designed to make common analysis tasks simple, like plotting via tools such as the mplhep package. Hist and Aghast will be initially driven by HEP (High Energy Physics and Particle Physics) needs, but outside issues and contributions are welcome and encouraged.

Python 3.8 is out, with new features and changes. The themes for this release have been performance, ABI/internals, and static typing, along with a smattering of new syntax. Given the recent community statement on Python support, we should be staying up to date with the current changes in Python. As Python 2 sunsets, we are finally in an era where we can hope to someday use the features we see coming out of Python release again!

For High Energy Physics, the go-to framework for big data analysis has been CERN’s ROOT framework. ROOT is a massive C++ library that even predates the STL in some areas. It is¹ also a JIT C++ interpreter called Cling, probably the best in the business. If you have heard of the Xeus C++ Kernel for Jupyter, that is built on top of Cling. ROOT has everything a HEP physicist could want: math, plotting, histograms, tuple and tree structures, a very powerful file format for IO, machine learning, Python bindings, and more. It also does things like dictionary generation and arbitrary class serialization (other large frameworks like Qt have similar generation tools).

You may already be guessing one of the most common problems for ROOT. It is huge and difficult to install – if you build from source, that’s a several hour task on a single core. It has gotten much better in the last 6 years, and there are several places you can find ROOT, but there are still areas where it is challenging. This is especially true for Python; ROOT is linked to your distro’s Python (both python2 and python3 if your distro supports it, as of ROOT 6.22); but the common rule for using Python is “don’t touch your system Python” - so modern Python users should be in a virtual environment, and for that ROOT requires the system site-packages option be enabled, which is not always ideal. And, if you use the Anaconda Python distribution, which is the most popular scientific distribution of Python and massively successful for ML frameworks, the general rule even for people who build ROOT themselves has been: don’t. But now, you can get a fully featured ROOT binary package for macOS or Linux, Python 2.7, 3.6, 3.7, or 3.8 from Conda-Forge, the most popular Anaconda community channel! Many more HEP recipes have now been added to Conda-Forge, as well! ROOT now also provides a conda docker image, too!