Histogram Speeds in Python

Let’s compare several ways of making Histograms. I’m going to assume you would like to end up with a nice OO histogram interface, so all the 2D methods will fill a Physt histogram. We will be using a 2 x 1,000,000 element array and filling a 2D histogram, or 10,000,000 elemends in a 1D histogram. Binnings are regular.

1D 10,000,000 item histogram

ExampleKNLMBPX24
NumPy: histogram704 ms147 ms114 ms
NumPy: bincount432 ms110 ms117 ms
fast-histogram337 ms45.9 ms45.7 ms
Numba312 ms58.8 ms60.7 ms

2D 1,000,000 item histogram

ExampleKNLMBPX24
Physt1.21 s293 ms246 ms
NumPy: histogram2d456 ms114 ms88.3 ms
NumPy: add.at247 ms62.7 ms49.7 ms
NumPy: bincount81.7 ms23.3 ms20.3 ms
fast-histogram53.7 ms10.4 ms7.31 ms
fast-hist threaded 0.5(6) 62.5 ms9.78 ms(6) 15.4 ms
fast-hist threaded (m)62.3 ms4.89 ms3.71 ms
Numba41.8 ms10.2 ms9.73 ms
Numba threaded(6) 49.2 ms4.23 ms(6) 4.12 ms
Cython112 ms12.2 ms11.2 ms
Cython threaded(6) 128 ms5.68 ms(8) 4.89 ms
pybind11 sequential93.9 ms9.20 ms17.8 ms
pybind11 OpenMP atomic4.06 ms6.87 ms1.91 ms
pybind11 C++11 atomic(32) 10.7 ms7.08 ms(48) 2.65 ms
pybind11 C++11 merge(32) 23.0 ms6.03 ms(48) 4.79 ms
pybind11 OpenMP merge8.74 ms5.04 ms1.79 ms
[Read More]
Categories: Python  Tags: programming python 

Python 3 upgrade

About ten years ago, Guido Van Rossum, the Python author and Benevolent Dictator for Life (BDFL), along with the Python community, decided to make several concurrent backward incompatible changes to Python 2.5 and release a new version, Python 3.0.

[Read More]
Categories: python  Tags: programming python pyupgrade 

A simple introduction to asyncio

This is a simple explanation of the asyncio module and new supporting language features in Python 3.5. Even though the new keywords async and await are new language constructs, they are mostly1 useless without an event loop, and that is supplied in the standard library as asyncio. Also, you need awaitable functions, which are only supplied by asyncio (or in the growing set of async libraries, like asyncssh, quamash etc.).

[Read More]
Categories: Python  Tags: async programming python 

A little example of how asyncio works

This is a simple example to show how Asyncio works without using Asyncio itself, instead using a basic and poorly written event loop. This is only meant to give a flavor of what Asyncio does behind the curtains. I’m avoiding most details of the library design, like callbacks, just to keep this simple. Since this is written as an illustration, rather than real code, I’m going to dispense with trying to keep it 2.7 compatible.

[Read More]
Categories: Python  Tags: async programming python 

Basics of metaclasses

This is a quick tutorial over the basics of what metaclasses do.

The Metaclass

Metaclasses, while seemingly a complex topic, really just do something very simple. They control what happens when you have code that turns into a class object. The normal place they are executed is right after the class statement. Let’s see that in action by using print as our metaclass.

[Read More]
Categories: Python  Tags: programming python