Skip to content

Running only affected tests

Updated

Several tools run fewer tests after a change: some pick the tests a change could reach, from git and the import graph, and others reuse results they recorded. hushrun's test reuse is the second kind: it leaves out each test that already passed on the files it reads, and shares that across laptops, CI and agents' sandboxes.

ToolPicks bySmallest unitRemembers a passAcross machines
Jest --onlyChanged, --findRelatedTestsgit changes and the static import graphTest fileNoNo
Vitest --changed, vitest relatedChanged files and static importsTest fileNoNo
pytest-testmonWhich code each test ran, through Coverage.pyTestYes, in .testmondataThrough its tmnet service
go testThe test binary, the files tests open and the variables they readPackageYes, on the machineNo
Nx affectedgit changes and each task's inputsTask, such as ui:testWith Nx's cacheWith Nx Cloud
BazelEach target's declared inputsTest targetYesWith a remote cache
Develocity Predictive Test SelectionA model trained on past changes and resultsTestIt weighs past resultsYes
hushrun tests: reuseThe files each test file, or each test, read when it passedTest, or test fileYesThrough your organization's backend

Tools that pick from a change

  • Jest. --onlyChanged runs the tests related to changed files in a git or Mercurial repository, and needs a static dependency graph, with no dynamic requires. --changedSince compares with a branch or commit, and --findRelatedTests takes a list of source files, which suits a pre-commit hook.
  • Vitest. --changed runs the tests affected by changed files, and vitest related the tests that cover a list of source files. Both follow static imports, not dynamic ones.
  • Nx. nx affected runs each task whose inputs a change touches, and the tasks that read their outputs. It selects whole tasks, such as a project's test, not single tests. In GitHub Actions it needs fetch-depth: 0.
  • Turborepo. turbo run --affected compares main with HEAD and, by default, runs every task of each package with a changed file. With too shallow a clone, every package counts as changed.

These need no record of earlier runs, so they work on a fresh clone. But they don't know what passed: a related test runs again even when it passed on the same code an hour ago, on another machine.

Tools that remember results

  • pytest-testmon. testmon records which code each test ran, with Coverage.py, and on the next pytest --testmon selects only the tests affected by changes. Its data lives in .testmondata, or in its tmnet cloud storage with --tmnet. It doesn't track static files such as text or XML, or services over the network, and it always reruns tests that failed last time.
  • go test. In package list mode, such as go test ./..., Go caches a package's successful result and replays it when the test binary and the flags match, and the files the tests opened in the module and the environment variables they read haven't changed. -count=1 turns it off. It's on one machine.
  • Bazel. With --cache_test_results=auto, the default, Bazel reruns a test only when it or its dependencies changed, it's marked external, --runs_per_test asks for more runs, or it failed. With a remote cache, results are shared. Every test target needs its inputs declared in BUILD files. See hushrun and Bazel.
  • Develocity Predictive Test Selection. An add-on of Develocity for Gradle and Maven tests on the JUnit Platform. A model trained on your builds' changes and results picks the tests likely to matter, at a level of confidence you choose.

How hushrun's test reuse differs

With tests set in hushrun.yaml, hushrun records the files each test file, or each test, read when it passed. After a change, it leaves out the tests whose files didn't change, and runs the rest:

hushrun: unit: 3 of 101 tests run, in 2 of 2 test files; 98 already passed on the files they read, and 2 more that already passed run anyway, as a check
  • Shared. A pass recorded on a laptop, in CI or in an agent's sandbox lets the others leave that test out, through your organization's backend.
  • No graph to describe. It works from what each test read, so there are no BUILD files and no task inputs to declare.
  • Many runners, named. pytest, Vitest, Jest 29 and 30, Go, Gradle (Java, Kotlin, Android's local unit tests), Maven, swift test and CTest, test by test from hushrun 0.16.0; whole test files for pytest, Vitest and Go from 0.13.0. Some need one line in their config: What your runner needs. Rust's cargo test reuses whole test binaries.
  • Safety checks. A small share of the tests it could leave out run anyway, and a test that fails though it passed on the same files suspends reuse for the check until a person resumes it (What keeps it safe).
  • First the whole check. Before any test, a check whose files all passed already skips as a whole (How it works).

Where others are better:

  • Environment variables. go test's cache counts the variables tests read. hushrun doesn't: a test that depends on one needs a check name per variant.
  • A change outside the sources runs everything. A README, a lockfile or a CI file may be read by any test, so changing one runs them all, unless you list it in tests.ignore (The trades). Jest and Vitest follow imports, so a file no test imports selects no test.
  • Off by default, per check. Start each check in observe and read what it would have left out before you turn on reuse.
  • Some set-ups can't be watched, such as Vitest with coverage on, or Jest with coverage on. hushrun then runs every test, and says why (Limits).

Which to choose

  • While you work on one machine, Jest's --onlyChanged, Vitest's --changed or pytest-testmon give fast feedback with nothing to set up beyond a flag or a plugin.
  • In a monorepo already on Nx, Turborepo or Bazel, use their affected runs and caches. hushrun can wrap the whole run, so code that passed runs nothing at all.
  • When laptops, CI and agents run the same suite, hushrun's test reuse shares what passed between them, for the runners it supports.

For the setup per runner, see Run only the tests a change touches.

Sources

Checked on 2026-11-14:

Markdown