Cache test results across machines
Updated
Test runners on their own don't reuse a pass: their caches make the next run faster, but the tests still run. To reuse a result on another machine you need a build tool that models your tests, or a wrapper that recognizes the code.
| Tool | What it reuses | Across machines | What it asks of you |
|---|---|---|---|
| Vitest | Nothing: - picks tests, and its cache orders them | No | Nothing |
| Jest | Nothing: - picks tests | No | A git or Mercurial repository |
| Gradle | A Test task's outputs, with - | With a remote build cache | A Gradle build |
| Bazel | Test results | With a remote cache | BUILD files for each package |
| Nx | Task results | With a remote cache | An Nx workspace |
| Turborepo | Task results | With a remote cache | turbo. and package. scripts |
| hushrun | A check's pass | With a shared backend | One wrapper around the command |
Test runners pick tests
Vitest's --changed runs the tests affected by changed files, and Jest's --onlyChanged does the same from version control. Both help while you work. They still need something to compare against, and neither records that a run passed. Vitest's cache puts failed and slow files first.
Build tools reuse results
A build tool that knows a test's inputs can decide it needn't run.
- Gradle.
Testis cacheable: with--build-cache, Gradle reuses its outputs instead of running it. Environment variables aren't tracked unless you declare them. - Maven. The build cache extension, for Maven 3.9 and later, keys on sources, the project model, plugins and their parameters.
- Bazel. With
--cache_test_results=auto, the default, a test reruns when it or a dependency changes, when it's markedexternal, with--runs_per_test, or after a failure. See hushrun and Bazel. - Nx and Turborepo. Both hash a task's inputs and replay its results, from a cache you can share. See hushrun and Nx Cloud and hushrun and Turborepo.
These are precise: change one package, and only what depends on it runs. The cost is adopting the tool, and describing your project to it.
A wrapper reuses a pass
hushrun run needs no description of your project. It wraps the command you already have, in any language:
"scripts": {
"test": "hushrun run -- vitest run"
}hushrun run unit -- cargo testIts key is the repository's files, the check's name, the exact command and the directory. Results go to a backend your laptops, CI and coding agents share, so a pass in one place skips the check in the others.
The trade is precision. By default a change anywhere in the repository runs every check again, until you split it into scopes. It also restores no outputs: a skip means the command didn't run, so a coverage report or a test log isn't there. Run with --force when you need them.
What makes a cached result wrong
Every one of these tools can reuse a result it shouldn't when a test depends on something outside its key:
- Environment variables and secrets. Nx, Gradle and Turborepo count them when you declare them, apart from framework prefixes Turborepo infers, such as
NEXT_PUBLIC_*. hushrun doesn't count them. - The runtime or operating system. Nx hashes the OS and CPU. For hushrun, give each variant its own name, such as
unit-node22andunit-node24. - Services and the network. A test that talks to a real database depends on that database. Keep such a test out of the cache, or expect its pass to mean less.
- Partial runs.
vitest run src/cart.test.tsisn'tvitest run. hushrun treats them as two checks, so a partial pass never stands in for a full one.