Introducing repo-review

I’ve released a new1 toolkit for running checks, similar to Ruff and Flake8 but designed to check configuration, called repo-review. It requires Python 3.10+2 to run and has no built-in checks, but is easy to write plugins for. A set of checks based on the Scientific Python Development Guide (which I also have a post about!) are available as a plugin, sp-repo-review. You can run repo-review in WebAssembly (via Pyodide), or in pre-commit, or as a GitHub Action. It supports multiple output formats, including Rich, HTML, and JSON. The system is based on fixtures (like pytest) and topologically sorts requirements. You don’t need to depend on repo-review to add a repo-review plugin. You can see a live version using sp-repo-review in-place here or standalone here.

Repo-review output example

Installing repo-review

Using repo-review is easy; the only complication is adding plugins. In order to run repo-review, you need at least one plugin. sp-repo-review is a first-party plugin based on the Scientific Python Development Guide, so I’ll demonstrate with that.

Repo-review provides an example webapp that you can use. See the example live here. Currently you can copy this webapp javascript file anywhere, and use something like the provided index.html to run it. It’s also easy to write your own, as repo-review also now supports writing directly to html (the webapp uses React).

This webapp can be embedded into an existing webpage if you set header={false}. You can set your own deps with deps = {["...", "..."]}. See more, including how to write your own, in the docs.

You need to add your plugin to the repo-review environment; with pipx, you can use inject to do this:

pipx install repo-review[cli]
pipx inject repo-review sp-repo-review
repo-review

However, if the plugin explicitly depends on repo-review, you can also just do:

pipx install sp-repo-review[cli]
repo-review

(sp-repo-review even supports pipx run sp-repo-review)

You can also use pre-commit:

- repo: https://github.com/scientific-python/repo-review
  rev: <version>
  hooks:
    - id: repo-review
      additional_dependencies: ["sp-repo-review==2023.07.13"]

You can use repo-review directly in GitHub Actions, as well:

- uses: scientific-python/repo-review@<version>
  with:
    plugins: sp-repo-review

This will provide a nicely formatted summary.

Learn more about installing in the docs.

Using repo-review

Repo-review has output that looks like this (HTML output shown; JSON, SVG, and rich terminal output supported too):

RF002 Target version must be set
⚠️ RF003 src directory specified if used
RF101 Bugbear must be selected

Must select the flake8-bugbear B checks. Recommended:

select = ["B"]  # flake8-bugbear

Every check either passes, fails, or is skipped. Checks can depend on each other - if one check depends on a failed or skipped check, it is skipped. Checks have an optional URL; if present, it will link to that URL. Failures have a markdown-formatted failure message.

Not shown above, but besides the error code, every check has an associated family, and the report is grouped by family.

Repo-review supports configuration in pyproject.toml (or on the command line):

[tool.repo-review]
select = [...]
ignore = [...]

You can list checks to include or ignore here, much like Ruff or Flake8.

Learn more about the CLI or programmatic usage in the docs.

Writing a plugin

Repo review is built around several core concepts. These are all tied to entry-points; a plugin can tell repo review where to find the following items by adding an entry-point. Read more in the docs.

Also, like flake8, you can implement all of these without importing repo-review. It’s easy to make an existing package be a repo-review plugin without adding an extra dependency.

Fixtures

The core item in repo-review is a fixture - very much like a pytest fixture. It looks like this:

def pyproject(package: Traversable) -> dict[str, Any]:
    pyproject_path = package.joinpath("pyproject.toml")
    if pyproject_path.is_file():
        with pyproject_path.open("rb") as f:
            return tomllib.load(f)
    return {}

There are two special, built-in fixtures: root and package, which represent the path to the repository root and package root, respectively. All other fixtures are built from these. There are two more built-in fixtures, as well: pyproject (above) and list_all. Fixtures can request other fixtures (pyproject, above, uses package). Repo-review will topologically sort the fixtures and compute them once.

Read more in the docs.

Checks

The core of repo-review are checks, which are designed to be very easy to write:

class PY001:
    "Has a pyproject.toml"

    family = "general"

    @staticmethod
    def check(package: Traversable) -> bool:
        """
        All projects should have a `pyproject.toml` file to support a modern
        build system and support wheel installs properly.
        """
        return package.joinpath("pyproject.toml").is_file()


def repo_review_checks() -> dict[str, PY001]:
    return {"PY001": PY001()}

A check is a class with the following:

  • Docstring: the check summary.
  • family: the family the check belongs to.
  • url (optional): The URL for more info about the check.
  • requires (optional): A set of check names to require.
  • check(): A function that takes fixtures and returns:
    • True: the check passed (or empty string).
    • False: the check failed. The docstring is the check error message (markdown formatted).
    • None: the check skipped.
    • A non-empty string: a dynamic error message.

One common trick is to make a base class, set the family there, and then use:

class General:
    family = "general"


class PY001(General): ...


def repo_review_checks() -> dict[str, General]:
    return {p.__name__: p() for p in General.__subclasses__()}

This allows you to skip listing every check multiple times. You can ask for fixtures in this function too, which allows you to dynamically configure or select checks!

Read more in the docs.

Families

You optionally can pretty-print and sort families by providing a mapping with nice names and an integer order:

def get_familes() -> dict[str, dict[str, str | int]:
    return {
        "general": {
            "name": "General",
            "order": -3,
        }
    }

Families are sorted by lower-order first, then by the key alphabetically. This function does not currently take fixtures.

Read more in the docs.

Future of repo-review

I’d like to work on better packaging and distribution for the javascript parts of the WebApp - currently it’s an in-browser babel file that you vendor. I’d like to provide a way to add descriptive output in addition to checks, so you could print information about the repo (like what backend was used, what license was used, etc).


  1. This was originally developed integrated with sp-repo-review, so it’s been around for about a year, but it wasn’t generalized and split apart until the Scientific-Python Developers Summit 2023. ↩︎

  2. This could, with some work – quite a bit of work, actually – be made available for Python 3.9. However, is is much easier to write checks for 3.10+, so most plugins would probably want to be 3.10+ anyway. ↩︎