How Rig works#
The other pages of this site describe what Rig does. This section describes how: the two components, the boundary between them, and where each guarantee lives in the code.
Two components, one rule#
Rig is a Go binary driving a pinned Clojure kernel:
┌─────────────────────────────┐ ┌──────────────────────────────┐
│ rig (Go binary) │ JSON │ resolver kernel (Clojure) │
│ CLI · lockfile · cache · │◄───────►│ pinned uberjar, one-shot │
│ hashing · classpath · │ stdin/ │ EDN · tools.deps · │
│ JVM launch · orchestration │ stdout │ tools.build │
└─────────────────────────────┘ └──────────────────────────────┘
│ os/exec
▼
java (project JVM, classpath from the lock)- The Go binary owns everything that touches bytes: the lockfile
(JSON), the content-addressed artifact cache, fetching and
sha256-hashing every artifact, classpath assembly, and JVM launch. It
never parses
deps.ednfor resolution — it reads manifests only to discover:auth :oidcrepositories and module paths, plus small literal CLI arguments. - The kernel is the EDN authority and the resolution engine: it
parses manifests, edits them (format-preserving), resolves dependency
graphs with
clojure.tools.deps, and builds jars withclojure.tools.build. It is a one-shot process: one JSON request in, one JSON document out.
The rule that separates the two: the kernel resolves, Go pins. The
resolve response of the kernel contains no sha256 at all. Go fetches
(or imports) every artifact, hashes the exact bytes it stores, and writes
the finished lock. The kernel never writes a hash, and never the lock
(the closed build).
The kernel protocol#
Go launches the embedded kernel jar with the request JSON on stdin. It reads one JSON document: the last line of stdout. Ops that spawn subprocesses can print noise before it. Diagnostics go to stderr:
// request
{"op": "resolve", "workspace": "/path/to/ws", "lock": "deps.lock",
"args": {"offline": false, "force": false}}| Kernel exit code | Meaning |
|---|---|
| 0 | ok — response JSON on stdout |
| 1 | the op failed (stack trace on stderr) |
| 2 | bad request (unknown op, malformed JSON, missing request) |
| 3 | a manifest still uses legacy keys — run rig migrate |
Exit 3 is the legacy guard: every op that reads a manifest refuses a
workspace that still carries pre-migration keys
(:exoscale.project/*, :exoscale.deps/*,
:slipset.deps-deploy/*, antq.core/*). The kernel has no legacy
fallback, so an un-migrated workspace cannot resolve, build, or publish;
migrate is the one exempt op.
Go also sets a small environment for the kernel JVM:
CLOJURE_CLI_ALLOW_HTTP_REPO=1— this lets:mvn/reposuse plainhttp://URLs (local dev servers, tests).RIG_REPO_TOKENS— an EDN{repo-id bearer}map for:auth :oidcrepositories. The kernel uses it for its own metadata probes.RIG_PROXY_REPOS— for the same repositories, the base URL of the local loopback auth proxy of Rig.tools.depscannot receive a bearer token of its own. So Rig rewrites the resolution traffic for those repos to the proxy, and the proxy forwards to the real URL with the bearer attached. Rig keeps the repo ids, so the lock attribution does not change.
Hot vs cold paths#
Hot commands read only the lock: they resolve nothing and need no network when the cache is warm:
| Command | What the kernel contributes |
|---|---|
run, nrepl, exec |
nothing — pure Go |
clean |
nothing — no lock needed at all |
test |
its jar, as the last classpath entry: it carries rig.runner, the exec-fn launcher that runs the module's test alias. The project's own Clojure comes first and wins on conflicts |
check (stage 3) |
the kernel computes the AOT plan (aot-plan op), then its jar as the last classpath entry: the runner's aot mode preloads the plan's requires and AOT-compiles the module's namespaces into a fresh class dir that leads the locked base classpath. Stage 2 (lock vs manifests) is the cold kernel op; stage 1 (artifact hashes) is pure Go |
fmt |
its jar, run directly: a pinned cljfmt bundled inside it |
Cold commands run a kernel process:
| Command | Kernel op |
|---|---|
lock |
resolve |
add / remove / update |
edit-dep, then resolve |
check |
check — read-only consistency report (stage 2); stage 3 also takes the aot-plan op |
tree |
tree |
outdated |
outdated |
new / new-module (test-runner pin) |
latest — newest release past the cooldown |
build |
build — tools.build inside the kernel JVM, on the locked classpath |
publish / install |
publish — POM + deploy target only; Go does the transfer |
release |
publish, orchestrated by Go over git |
migrate |
migrate |
A hot command that needs the network needs it only to fetch bytes the lock already pins (the closed build).
The kernel ships inside the binary#
The kernel jar is itself a supply-chain item, so it ships embedded in the
rig binary (go:embed): the kernel uberjar, the rig.runner jar, and the
vendored nREPL jar. On first use Go extracts each jar into the store at
its content-sha path (~/.local/share/rig/kernel/<sha>/, …/runner/<sha>/,
…/nrepl/<sha>/), adopting a cached copy when its digest matches. A
read-only store works from the second run on: the first run extracts the
jars, and nothing is fetched or written afterwards. There is
no download URL and no hash pin to maintain: what the binary carries is
what runs. The kernel's identity (coordinate, version, git sha) is stamped
into the binary at release time via -ldflags, the same way as the rig
version.
The lock records the kernel that produced it in its resolver block: the
identity plus the sha256 of the jar that actually ran — the digest of the
embedded jar, or the file hash when a RIG_KERNEL_JAR dev override is in
play. Go stamps it at lock time; the kernel never stamps it. So a lock
always says who resolved it.
The jar is not byte-reproducible by default, and nothing in the flow
needs it to be. Go stamps every other hash in the system this way: the
hash in the lock, the SHA256SUMS entries. A workspace that wants
byte-reproducible builds opts in with :rig/timestamp-string. With it
set, every entry of the built jar and uberjar carries the pinned
timestamp, and Go writes the entries in name-sorted order. So the same
sources, lock and build JVM (:rig/jvm) produce a byte-identical jar. The
sha256 of the jar then becomes a stable fingerprint of the sources, lock
and build config. Zip timestamps have a 2-second resolution — the limit of
the format. Rig builds the runner jar (the rig.runner class files only)
the same way; it is embedded and materialized like the kernel jar.
Checkpoint/restore acceleration#
Restore-eligible kernel ops can run on a CRaC checkpoint of a warmed
kernel instead of a cold JVM fork. Restore takes ~0.3 s against 1–3 s for
the cold fork. Every restored run re-dumps the image, so the image stays
current. The whole path lives in internal/kernelrun. The package holds
the image store, the restore/cold selector inside kernel.Call, and the
bootstrap that forks and parks the checkpoint JVM. An image key pins the
kernel jar bytes, the JVM build, the engine, and the CPU flags. The
JVM-side contract is load-bearing and documented in that package's header.
It covers the exact flag set and where the kernel parks
(rig.kernel.Bootstrap, via reflection so the jar builds on plain JDKs).
It also states why success is "dump-kill + image on disk, not exit code".
Refresh dumps are deltas referencing their parents, so a
chain is deletable only as a whole (rig crac clean). Rig pins the
managed Zulu CRaC JDK (rig crac install, jdk.CRaCPins) exactly: image
keys track JVM builds, so channel drift would strand every image.
Code map#
Go (rig/internal/)#
| Package | Job |
|---|---|
cli |
command implementations and hot/cold orchestration |
kernel |
the kernel and runner pins; one-shot invocation; fetch + verify into the store |
lockfile |
the lock JSON model, validation, staleness detection |
fetch |
artifact retrieval: cache → m2 → download, hash-verified |
cache |
the content-addressed store (artifacts/<sha256>) |
classpath |
lock → absolute JVM classpath |
digest |
file sha256 |
workspace |
workspace discovery (root, modules, lock path) |
jvm |
java discovery and launch |
jdk |
managed Temurin JDKs (Adoptium API, sha-verified) |
maven |
~/.m2/settings.xml: credentials, local repository |
oidc |
OIDC tokens for :auth :oidc repositories |
proxy |
the loopback auth proxy for kernel traffic |
ednlit |
the small EDN reader for CLI arguments and workspace manifests |
updater |
release checks and rig self-update |
Clojure kernel (resolver/src/)#
| Namespace | Job |
|---|---|
rig.resolver.main |
entry point: op dispatch, exit codes, legacy- and removed-key guard |
rig.resolver.manifest |
manifest read (byte sha256, legacy- and removed-key refusal, :rig/* accessors, module version) |
rig.resolver.resolve |
the resolve op: selection → create-basis → lock document |
rig.resolver.versions |
version candidates, cooldown selection, repo attribution, auth proxy |
rig.resolver.plan |
classpath roots → lock entries (mvn / git / local classification) |
rig.resolver.check |
the consistency report |
rig.resolver.edit |
format-preserving manifest edits (rewrite-clj + cljfmt) |
rig.resolver.tree |
dependency tree display |
rig.resolver.outdated |
what's newer upstream |
rig.resolver.aot |
the AOT plan: the non-project require closure (preloads), the module's namespaces in dependency order, and the effective entry points (the module's :gen-class namespaces, plus :rig/main when it names a non-project namespace with a source on the classpath) — drives the two-phase AOT compile of build and check (stage 3) |
rig.resolver.build |
jar/uberjar via tools.build from the locked classpath, with the two-phase AOT compile |
rig.resolver.publish |
POM + deploy target (plan-only) |
rig.resolver.migrate |
legacy workspace → :rig/*, in place |
rig.resolver.git-sync |
thread-safe wrapper around the tools.gitlibs cache |
rig.runner |
hot-path test/check launcher; its jar (embedded like the kernel jar) is appended to the project's locked classpath, never the kernel jar itself. Requires Clojure ≥ 1.8.0 on the module's locked classpath: the gen-class wrapper's static initializer calls clojure.lang.Util/loadWithClass(String, Class) and modern-compiled namespace bodies reference clojure.lang.Tuple, both absent from Clojure 1.7.x — check/test report clojure-floor below it |
rig.fmt |
the cljfmt entry point bundled in the kernel jar |
Where the guarantees live#
- Ecosystem-consistent version resolution — the kernel delegates
graph resolution to
clojure.tools.deps. It adds a selection layer for floating requirements and a comparison where a local coordinate beats a published one: version resolution. - A build closed to the lockfile — no EDN on the hot path, a validated lock, Go-computed hashes, gated sources: the closed build.
- No source next to its class — every jar the build produces drops
the bundled
.clj/.cljcthat sit next to their AOT__init.class.RT.loadloads the source instead of the class whenever the source is not strictly older. Sources exploded from dependency jars carry no such ordering. Equal timestamps would make the runtime recompile the namespace from the bundled source — two class identities. A source with no base__initis the only payload of the namespace, so the build keeps it. The same holds for a source whose only__initlives underMETA-INF/versions/(the fallback of the floor). The build refuses a dependency that ships a source strictly newer than its class. In that jar the source is what the runtime loads, and the build refuses to guess which of the two is the payload.