We’ll talk a little about each topic, but the main focus is showing how to add each of these processes to your code, and how they work.
In between each section, try adding the step we just discussed to your own code. If you run into issues or are confused, say so — we can then answer that question for the whole group.
This is ‘expected knowledge’ for software developers, but it’s rarely taught to scientists. Yet it’s exactly what keeps scientific code trustworthy: tests catch when a change silently breaks a result, and CI runs those checks automatically on every change.
Understanding testing and continuous integration (CI) well is also a highly transferable skill — it shows up in nearly every software engineering role.
Regardless of industry or academia, CI systems can run lots of different tasks — whole processing pipelines, docs builds, releases — and they’re free for open-source projects.
(Please use CI responsibly — these are shared, free resources.)
Every example below uses the same tiny package, rescale, which maps an array onto the interval [0, 1].
src/rescale/rescale.pyimport numpy as np
def rescale(input_array: np.ndarray) -> np.ndarray:
"""Linearly rescale an array to the range [0, 1].
Maps the minimum value of the input to 0 and the maximum to 1,
scaling all other values proportionally in between.
"""
L = np.min(input_array)
H = np.max(input_array)
output_array = (input_array - L) / (H - L)
return output_arraySmall enough to hold in your head, and buggy enough to be interesting — notice what happens when every input value is the same.
Testing consists of writing small functions that check:
Running your tests lets you check that new changes to the code base did not break anything in your code (at least, what you are testing for!).
As you write tests, you get to experience what it takes to use your library, and this might lead you to refactor parts of your code. Refactoring is an important part of a software life-cycle!
Tests are usually grouped by scope — how much of the system one test exercises:
You don’t need all of these on day one. For most scientific code, start with unit tests — they catch the most bugs for the least effort.
The smallest scope: test one function or class on its own.
# tests/test_rescale.py
import numpy as np
from rescale.rescale import rescale
def test_rescale() -> None:
"""Test that rescale works correctly for a simple case."""
input_array = np.array([1, 2, 3, 4, 5])
output_array = rescale(input_array)
expected_array = np.array([0, 0.25, 0.5, 0.75, 1])
np.testing.assert_allclose(output_array, expected_array)Bigger scope, fewer of them:
These catch bugs unit tests can’t — wrong assumptions about how components fit together — but they’re slower and harder to debug.
This is the test pyramid: many fast unit tests at the base, fewer integration tests above, and only a handful of slow end-to-end tests at the top.
Grouped by purpose, not scope: a regression test makes sure a bug that was fixed stays fixed.
The workflow when you find a bug:
Our rescale divides by zero when every input value is equal (H == L). A regression test would feed it a constant array and assert it behaves sensibly — so a future change can’t silently reintroduce that crash.
Instead of hand-picking inputs, state a property that must always hold and let a tool generate many inputs trying to break it.
This shines for scientific code, where outputs obey mathematical rules:
rescale’s output should always lie in [0, 1]See Hypothesis for the Python implementation.
Rather than copy-pasting a test for each case, hand pytest a list of them:
import numpy as np
import pytest
from rescale.rescale import rescale
@pytest.mark.parametrize(
"input_array, expected_array",
[
(np.array([1, 2, 3, 4, 5]), np.array([0, 0.25, 0.5, 0.75, 1])),
(np.array([5, 4, 3, 2, 1]), np.array([1, 0.75, 0.5, 0.25, 0])),
],
)
def test_rescale_parameterized(
input_array: np.ndarray,
expected_array: np.ndarray,
) -> None:
"""Test that rescale works correctly for multiple cases."""
output_array = rescale(input_array)
np.testing.assert_allclose(output_array, expected_array)A fixture loads a resource once and hands it to any test that asks for it by name — useful when setting the resource up is slow.
Sometimes the correct behavior is an exception:
Configure it once in pyproject.toml so everyone runs the same thing:
MORE IMPORTANTLY
black is the long-standing default; ruff format is a fast, near-drop-in replacement that also does your linting.
The formatting, linting, and type-checking hooks are tied together by pre-commit:
Install it once per clone:
After pre-commit install, the hooks run automatically on every git commit, on the changed files. To run them on demand across the whole repository:
Warning
Without pre-commit install, committing won’t trigger any of these checks — they’re silently skipped. (Invoking black / ruff / mypy directly still works, but the git hooks won’t fire on commit until you set this up.)
Formatting focuses on directly changing the code to some “standard” style. Linting looks for common problems in the code.
A nice example of the difference: formatting won’t change the following,
but linting should inform you (if not automatically fix) that foo from package_a is imported twice, and that a nicer import block looks like:
I won’t list all the rules it checks against, but there are a lot.
# pyproject.toml
[tool.ruff]
line-length = 88
target-version = "py311"
extend-select = [
"E", # style errors
"F", # flakes
"D", # pydocstyle
"I001", # isort
"UP", # pyupgrade
"N", # pep8-naming
"B", # flake8-bugbear
"RUF", # ruff-specific rules
]
extend-ignore = [
"D100", # Missing docstring in public module
"D107", # Missing docstring in __init__
]Type checking is the most thorough analysis of your code you can do before you even run it.
Python added optional types in version 3.5. At first they were simply for analysis of the code prior to running, to check for even more bugs and possible error cases, but in recent versions they are also being used to speed up your programs.
They can be annoying to add and work with — very annoying, because they are optional in the language, so there are a lot of hacky ways they can be manipulated.
Typing is handled by annotations in the code:
import random
# yay only a single value
var_int: int = 5
var_float: float = 0.5
var_bool: bool = False
var_string: str = "wow!"
# oh no multiple values
list_of_strings: list[str] = ["hello", "world"]
list_of_mixed: list[str | int | bool] = ["hello", 3, True]
# lists don't have any constraints on size but tuples do
tuple_of_two_values: tuple[str, int] = ("number", 10)
tuple_of_n_values: tuple[str, ...] = tuple(
["python why did you add this" for i in range(random.randint(1, 5))]
)
# sub-objects
dict_of_dicts: dict[str, dict[str, str | int]] = {
"alice": {"name": "Alice Smith", "age": 30},
"bob": {"name": "Bob Boberson"},
}from abc import ABC, abstractmethod
from typing import Type
class Localizer(ABC):
@abstractmethod
def localize(self, text: str) -> str:
raise NotImplementedError()
class EnglishLocalizer(Localizer):
def localize(self, text: str) -> str:
return "Hello world"
localizers = {
"en": EnglishLocalizer,
}
def get_localizer_not_initialized(lang: str) -> Type[Localizer]:
return localizers[lang]
def get_localizer_initialized(lang: str) -> Localizer:
return localizers[lang]()You rarely have to hand-write hard types — libraries ship their own.
numpy, pandas, and most scientific libraries provide types you can reuse directly in your annotations:
dataclassA deeply nested dict[str, dict[str, ...]] is technically correct but hard to read, and the checker can’t tell you which keys must exist.
from dataclasses import dataclass
# Technically correct, but the checker can't catch a misspelled key:
dict_of_dicts: dict[str, dict[str, str | int]] = {
"alice": {"name": "Alice Smith", "age": 30},
"bob": {"name": "Bob Boberson"},
}
# Define the sub-object once, with named + typed fields:
@dataclass
class PersonDetails:
name: str
age: int | None = None
people: dict[str, PersonDetails] = {
"alice": PersonDetails(name="Alice Smith", age=30),
"bob": PersonDetails(name="Bob Boberson"),
}
# The payoff: attribute access is typed and checked
# (autocompletes, and a typo like .agee would be flagged)
people["alice"].ageShip a py.typed marker file inside your package so downstream users get your type information too.
It is hard to learn, but do practice using it. It will make writing code easier the more you do it, as it will check a lot of assumptions for you.
Some resources to continue learning:
Continuous integration (CI) is meant to reduce or remove bugs entering code over time — whether the code base is changing (new features, bug fixes) or upstream dependencies are changing (new releases).
The main idea: by checking your tests, formatting, linting, and types for each commit or PR, you know that nothing is breaking.
If something does break, you know the exact commit or PR which broke it.
Further, it lets you test on more machine setups than your own. For example, you might have a Linux laptop and a Windows desktop, using Python 3.11 on both — but have users on macOS with Python 3.10.
CI systems (GitHub Actions, GitLab CI/CD, Azure Pipelines, etc.) let you run the same suite of tests across all of these machine setups.
Fortunately, once you have testing, formatting, linting, and type-checking set up locally, it is pretty easy to add CI to your repository!
# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
tags: ["v*"]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]
os: [ubuntu-latest, macOS-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: pip install .[test]
- run: pytest tests/The same pre-commit command you run locally — so CI can never disagree with your machine about what “clean” means.
Don’t cram everything into one workflow. Split them by trigger and purpose:
ci.yml — tests and lint. Runs on every push and pull request, so you get feedback on every change.docs.yml — builds and deploys the documentation. Runs only on push to main, and only when the docs sources change (a paths: filter).Why separate them?
main only.paths: filters skip work the change can’t affect.pages: write / id-token: write; testing doesn’t. Keeping them apart limits what each job can do.Once you have tests, a natural question is: how much of my code do they actually exercise?
Code coverage measures which lines (and branches) of your source code are run when your test suite executes. It is usually reported as a percentage:
if/else branches were takenA line that is never run by any test is a line whose behavior you are not verifying.
High coverage is not the same as good tests.
Use coverage as a guide to find untested code, not as a goal in itself.
We measure coverage with pytest-cov, a plugin built on coverage.py:
--cov=rescale measures the rescale package, and --cov-report=term-missing prints which lines were not covered.
Add a coverage job so every commit and PR reports it. It generates a machine-readable report and uploads it to a service such as Codecov, which then comments on each PR showing how coverage changed:
Uploading to Codecov needs a CODECOV_TOKEN repository secret, which a repository admin configures from the Codecov dashboard.
A release job lives alongside the test, lint, and coverage jobs, but it runs only when you cut a release — typically triggered by a version tag (e.g. v0.1.0) or a published GitHub Release, not on every commit.
release:
needs: [test, lint]
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write # required for PyPI Trusted Publishing (OIDC)
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build distribution
run: |
pip install build
python -m build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1Authentication uses PyPI Trusted Publishing: PyPI trusts your GitHub repository directly, so there is no API-token secret to manage or leak.
Everything above lives in one repository, and a task runner such as just gives it a single front door:
# Justfile
install:
pip install -e .[test,dev,docs]
test:
pytest tests/
lint:
pre-commit run --all-files…and CI runs the same three commands on every push and pull request.
Much of the written content came directly from pydev-guide.github.io.
It is still under development, but some tutorials are already available. Highly recommend starring it and checking back every few months.
These materials are adapted from lessons created by Eva Maxfield Brown for the URSSI Winter School, and updated for the June 2026 school.