Share compiled code

With your organization's shared store on, Rust crates that the repository's trusted CI compiled are reused by teammates' laptops, coding agents' sandboxes and the repository's other CI jobs. This page sets it up in the order that works. How it works, and whose code each machine reuses: Compiled code.

What to expect

• Reuse runs one way. Laptops, sandboxes and pull requests reuse what the repository's trusted CI built. CI never reuses what a laptop built. A person's laptop shares, by default, only with that person's other machines: every machine signed in to this organization as them. Teammates share with each other only where an admin chooses One for the team.
• The same system, the same compiler. A machine reuses only code built for its own system and compiler version, so a Mac gets nothing from Linux CI, and the reverse. Pin one toolchain, with rust-toolchain.toml, on laptops and CI.
• No other compiler wrapper. A job or a machine that already sets RUSTC_WRAPPER, as sccache does, turns hushrun's build reuse off for that run, and the run says so in one line. Remove it where commands run through hushrun.
• Offline work stays local. What a machine builds with no network is kept on that machine and never uploaded later.
• GitLab: the shared store doesn't work for GitLab CI yet. Laptops and sandboxes of a GitLab repository still share between themselves.

Before you begin

• Every machine and CI job runs hushrun 0.15.0 or later. A repository keeps the version it pins until the pin moves: "hushrun": "^0.13.0" in package.json stays on 0.13, and a workflow that passes version: to hushrun/hushrun/action/setup stays on that version. The setup action with no version: installs the newest release. Move the pins, and commit the lockfile.
• You're an admin or owner of the organization.
• The repository is connected with npx hushrun init, and its CI runs on GitHub Actions with id-token: write, which init writes.
• The Rust commands you want faster run through hushrun as hushrun run NAME -- COMMAND, such as hushrun run test -- cargo test, where NAME is the check's name. Only that form shares: the checks of hushrun.yaml, run as hushrun run CHECK, reuse only what a machine compiled itself.

The dashboard's Shared store page, https://app.hushrun.com/<org>/-/build, holds Sharing (the switch, what machines do, and the pool), the list of Repositories, Suspensions and disagreements, Agent tokens, Usage and Delete everything. Each repository has its own page, which its row in Repositories opens: its Confirmation, Trusted branches, its runs and its Suspensions. Members see both pages and change nothing.

Confirm the repository

A repository shows up under Repositories as seen once one of its GitHub Actions jobs has run hushrun with its ID token: hushrun has had a verified token from its CI. Open its page from its row, and choose Confirm… under Confirmation.

To confirm is to check the repository's and its owner's ids on GitHub, as its CI's tokens carried them, and pin them. If the repository is later transferred, or deleted and created again under the same name, its CI is no longer trusted and its code is no longer used. Confirming it again starts with no trusted branches.

Until a repository is confirmed, its CI shares nothing and isn't touched, and sandboxes have no trusted code of it to reuse. You can confirm repositories and list branches while sharing is off.

List its trusted branches

A trusted branch's CI writes the code everyone working on the repository reuses, so list only branches whose pushes you trust, usually main, and release branches if you build releases from them.

On the repository's page, under Trusted branches, choose from Branches its CI was seen on, most runs first, and List… each one. A branch shows up there only once its CI has run hushrun, so a branch can't be listed ahead of time. Branches are listed by their exact names, one at a time: there are no patterns, and tags can't be listed (why).

The page warns beside a listed branch that GitHub doesn't protect: anyone with write access can push to it, and so write code the repository reuses. Protect it on GitHub, or don't list it.

A branch you list stays trusted while you are an admin or owner of the organization. If you leave or lose the role, it stays trusted 14 days more and then stops, unless another admin chooses Confirm on its row. The same goes for three other settings admins give: trusting manual runs, trusting self-hosted runners, and an agent token's writing. See When an admin leaves.

Turn sharing on

On the Shared store page, under Sharing, choose Turn on…. The dialog says what it does before it's done:

• A member's machine takes part where that member turned build reuse on with hushrun build on, which is off by default. There it uploads what it builds inside hushrun run NAME -- COMMAND, unless the member keeps that machine local.
• The confirmed repositories' CI joins in. Other repositories' CI isn't touched.
• What is stored counts toward your plan's allowance: see Plans and limits.

Turning sharing on always starts by watching, also when it was reusing before it was turned off. Reusing is a step of its own.

On Enterprise, the shared store's allowance is set by contract. Until one is set, there is nothing to turn on, and a run says so:

hushrun: build: sharing is off for this run: the org's plan sets its shared storage by contract, and none is set yet: get in touch with hushrun to set it

Watch, then reuse

While sharing is watching (the dashboard's What machines do: Watch first), machines compile everything as usual. They store and upload what they build, and compare the crates they compiled with the shared store's copy. Nothing downloaded is reused yet.

The page's Usage panel shows the comparisons by who made them, trusted CI first, since a member's machine could report anything. Once trusted CI has compared crates and none differed, an admin chooses Start reusing…. Nothing moves to reusing by itself, and you can go back to watching at any time.

Note: Watching is the organization's mode. It isn't hushrun build observe, which is build reuse on one machine, nor hushrun build share observe, which is that machine's own sharing. They are three different switches.

Turn on build reuse on your machine

Each person turns build reuse on once per machine. It's off until then, and a machine with it off takes no part in the shared store:

$ hushrun build on
Build reuse is on: commands run by `hushrun run` reuse compiled crates kept in /Users/ana/Library/Caches/hushrun/build.
A block not used for 15 days is deleted.

You need to be signed in to the organization, as npx hushrun init does. Then run your checks as usual. The run's last lines say what came from the shared store:

hushrun: test running: no pass yet for this exact code
…
hushrun: test passed in 25s; it won't run again on this code
hushrun: build: 226 of 360 crates reused (226 from the shared store), 134 built, saving about 2m 3s

While the organization is watching, nothing comes from the shared store, and one more line says how many crates your machine compared with it: build: sharing observed: 41 blocks compared with the shared store, none differed.

Keep your machine local

To use build reuse on your machine without sending or fetching anything:

$ hushrun build share off
Sharing is off here: nothing goes to or comes from the org's shared store. Reuse on this machine is as the mode says.

• hushrun build share observe uploads what you build and compares what you'd download, but reuses no download.
• hushrun build share on, the default, shares as the organization's setting says.
• HUSHRUN_BUILD_SHARE=off does the same for one command.

Your setting can lower the organization's, never raise it. hushrun build status shows it, and what sharing does here and why.

With sharing off on your machine, a run reuses nothing it only downloaded: it compiles those crates once, and they are then your machine's own. So the first build after hushrun build share off compiles what had been downloaded.

CI

Nothing new goes in a GitHub Actions workflow. Once the repository is confirmed and sharing is on, each job's hushrun run NAME -- COMMAND gets the shared store. A hushrun run CHECK in CI doesn't: it builds as it would without hushrun's build reuse.

• Each job keeps compiled code in a store of its own under its temporary folder, RUNNER_TEMP, which goes with the job. Nothing another job left on the runner is reused or uploaded.
• It asks as itself, with the job's ID token, so id-token: write must stay. Without it nothing is shared, whatever other token the environment holds, and the run says the job can't mint an ID token.
• A job on a trusted branch writes trusted code. Any other job writes only its branch's own code, or nothing: see Whose code a machine reuses. A job that changes a file git tracks before it builds, such as a version bump or a cargo update, writes no trusted code.
• When the command ends, the job waits for its uploads: at most 2 minutes for a job writing trusted code, at most 20 seconds for any other, such as a pull request's, and never longer than a quarter of the compile time they stand for.
• A job of an organization that doesn't share, or of a repository nobody confirmed, runs exactly as before and says nothing.
• A job that already sets RUSTC_WRAPPER, as sccache's setup does, shares nothing, and its run says so in one line: remove the wrapper from jobs that run through hushrun.
• A job outside GitHub Actions that has an agent token reads the repository's trusted code alone and uploads nothing, whatever that token may do elsewhere. A job with only a person's sign-in shares nothing.
• Caching cargo's registry (~/.cargo) is fine: hushrun reads what's there by its content.

Note: Don't cache target/ between jobs while the shared store is on. A restored folder proves nothing about how its files were built, so hushrun names what it finds there by its bytes and shares only what the job compiled itself. Crates built against a proc macro from a restored folder never go up.

When a job on a listed branch isn't trusted, its run says why, once. In hushrun's lines, a block is one compiled crate, and the repository's trusted blocks are its trusted code:

hushrun: build: this job reads the repository's trusted blocks and writes none: it names a deployment environment

Agents' sandboxes

A sandbox with HUSHRUN_AGENT needs nothing more. hushrun takes a machine where no person is signed in for a sandbox, and there build reuse is on by default whenever the organization shares. A written off wins: hushrun build off on that machine, or HUSHRUN_BUILD=off, turns it off.

• The sandbox reuses its repository's trusted code, so it reuses something only once the repository is confirmed, a trusted branch's CI has built, and an admin has chosen Start reusing.

• It writes nothing unless an admin turns writing on for its agent token, under Agent tokens on the Shared store page. Then it reuses and writes code that token's sandboxes share, or the team's pool with One for the team. Writing is off by default: one token often serves many sandboxes, and with writing on, one sandbox that goes wrong spoils the code all of them reuse.

• A sandbox that reaches only a list of hosts must allow the shared store's host, which ends in .r2.cloudflarestorage.com, beside api.hushrun.com. Without it, the sandbox compiles as usual, and the run names the host to allow, once:

  hushrun: build: the shared store's host, acme.r2.cloudflarestorage.com, can't be reached from here: where this network allows only some hosts, allow acme.r2.cloudflarestorage.com to reuse what other machines built

A sandbox that loses its network

Some sandboxes, such as Codex cloud's, turn the network off once setup ends. Fetch the compiled code during setup, while the network and HUSHRUN_AGENT are there, naming the checks the agent will run:

$ hushrun build fetch test
Fetched 212 compiled crates (480 MB) and 340 hints for later runs here, good for 24 hours. No token was kept.

• Each name is a check as hushrun run runs it: hushrun run test -- cargo test is test. With no name, it fetches the crates that the most recent run of any check of the repository on this system used.
• It keeps what it fetched sealed, and keeps no token. Hints in its line are the lists of files each crate's compile read, which tell a later run what to hash. After 24 hours it's all deleted.
• A later run uses it only for the same server, organization and repository, and only when hushrun's server can't be reached. Such a run uploads nothing and sends nothing. While the server answers, the run asks it, as any run does.
• The agent's later runs need no token, but they must name the same server: through the repository's package.json or hushrun.yaml, as npx hushrun init writes, or HUSHRUN_BACKEND. A later phase that has the network but no token is refused, and uses nothing from the fetch.
• A fetch made while the organization is watching reuses nothing.
• A fetch is not told about a withdrawal: what it holds is reused for the rest of its 24 hours.
• Run it at the repository's root, after the toolchain is installed, with HUSHRUN_AGENT, an agent token in HUSHRUN_TOKEN, or a person's sign-in. --format json gives blocks, bytes, hints, store and good_for_hours.
• In GitHub Actions, a job with an ID token keeps the fetch in its own store, for a later step of the same job that can't reach hushrun's server, such as a coding agent behind a firewall. Elsewhere in CI it refuses, since nothing lasts to the next step.

Name the variables your build reads

A build script or a proc macro can read an environment variable without telling the compiler. Its value then ends up in the compiled crate, but not in what makes two compiles the same. When the compiled files hold such a value of 8 bytes or more, hushrun keeps the crate on the machine and says so:

hushrun: build: 2 crates stay on this machine: their outputs hold the value of DATABASE_URL, which no key holds (a project names such a variable under build: env:)

Only the exact variables a CI system documents as its own, and a few the build tools and the shell set, are left out of this search. A project's own variable is searched even when its name starts like a CI system's, such as BUILD_ENV.

Name such variables in hushrun.yaml, at the top level:

build:
  env:
    - DATABASE_URL

Every compile of the repository is then keyed by their values, on every machine: a machine with another value compiles its own crate. The list changes no check's definition. hushrun can't see a value a macro transformed, such as a hash of it, so name every variable your build reads that way.

Naming a variable lets crates that hold its value go up, sealed with a key that includes the value: only someone with the source and that value can open them. Name a secret only if that is acceptable.

When a run says sharing is off

The run builds as it would without the shared store, and says why in one line:

A machine with no network at all says nothing about sharing during a run: it builds and reuses from its own store. A machine that takes no part says nothing either. hushrun build status says why, in its sharing line:

Every line: Lines hushrun prints.

Delete everything

To remove everything your organization stored:

1. On the Shared store page, under Sharing, choose Turn off…. Machines and CI stop downloading and storing at once, and no machine reuses anything it only downloaded from its next run on: each compiles those crates once, and they are then its own. CI and sandboxes start empty, so they reuse nothing more. Only a machine with no network reuses an older download, until it's back online.
2. At the bottom of the page, choose Delete everything. It shows how much goes first, then removes all compiled code, and the lists machines use to find it. Nothing brings it back.

The first build on each machine after sharing goes off compiles what it had downloaded.

Turning sharing off alone deletes nothing: stored code stays until it goes unused for 30 days, and still counts toward the allowance, so turning sharing on again finds it.

Smaller removals each say how much goes before it goes: withdrawing a branch, a run or a commit, taking a confirmation back, turning a token's writing off, switching the pool, and removing a member, which deletes the code their machines stored.

hushrun build clean empties your own machine's store.

Special cases

GitLab CI

The shared store doesn't work for GitLab CI yet. Its jobs build as they do without it, and a GitLab repository has no trusted code for other machines to reuse. Its laptops and sandboxes still share between themselves: each person's machines reuse what that person's other machines built, or the team's pool does with One for the team.

Manual runs

A manual run (workflow_dispatch) on a trusted branch is trusted only when an admin chooses Trust manual runs… on the repository's page. It's off by default, since anyone with write access can start one, and a workflow that takes the code to build as an input would then write that code as trusted.

Self-hosted runners

A job on a self-hosted runner writes trusted code only when an admin chooses Trust self-hosted runners… on the repository's page. That trusts everything that outlives a job on those runners: the store, the toolchain, cargo's folder and hushrun itself. Choose it only when each runner is thrown away after its job, or runs only workflows you trust. It's refused for a public repository, and it ends after 90 days unless an admin confirms it again.

Merge queues and tags

A merge queue's jobs, and jobs on tags, read the trusted code and write nothing.

A bad run

On the repository's page, under its runs, an admin can withdraw a run, or a commit with every run of it. Its code is deleted at once on hushrun's server, and nothing it uploads later is accepted.

In an incident, in this order:

1. Withdraw the run, the commit or the branch, and leave sharing on, so machines hear the "no".
2. A machine that reaches hushrun's server stops reusing that code within a day: at its next run it asks again about every crate it downloaded whose last word from the server is over a day old. A machine with no network isn't told until it's back online: meanwhile it reuses a crate it downloaded only where its own earlier compiles lead to that crate. A sandbox's fetch made before the withdrawal is reused for the rest of its 24 hours.
3. To stop every download at once instead, turn sharing off: from each machine's next run, nothing only downloaded is reused (Delete everything).

When two trusted builds of the same crate disagree, that crate's stored copy is deleted and the crate is suspended in the repository for 7 days, or until an admin clears it under Suspensions: until then every machine of that repository compiles it.

When an admin leaves

Four settings hold only while the admin who gave them is an admin or owner of the organization: a trusted branch, trusting manual runs, trusting self-hosted runners, and an agent token's writing. A repository's confirmation, the switch and the pool don't lapse.

When that admin leaves or loses the role, each of the four holds 14 days more, and its row says until when. To keep it, another admin chooses Confirm on it: on the repository's page for the first three, under Agent tokens for a token. If nobody does, it is withdrawn: its code stops being used and is deleted, and listing it again starts empty. If the admin comes back, it holds again. Owners get an email the day the 14 days start.

Debugging with build reuse on

Inside hushrun run with build reuse on, a dependency's paths read /hushrun/cargo/… in panics and debug info, so that compiled code names no machine's folder. Where rustup's rust-src is installed, an error that points into the standard library shows /rustc/<commit>/… with no source line. A debugger needs a source map to open those files.

Next

• Compiled code: how it works, and whose code each machine reuses.
• Security and data: what leaves the machine, sealed.
• Agents in sandboxes: the sandbox's side.
• hushrun build: every command and setting.