Git on the bucket: what we learned from Piper, CitC, Enroute and Scion

The Git protocol post covered how stock git reaches a bucket: pkt-lines, protocol v2, remote helpers, and how git-remote-loams pushes by writing the next segment of a write-ahead log with a create-only PUT. This post is about the design behind it. Loams Git is meant to hold a monorepo that thousands of agents work in at once. That is a different problem from hosting repositories, and other people have solved parts of it at scale. We read four of them for one question each:

• Piper and CitC (Google): what is a workspace, if it is not a clone?
• Continuity (Cursor): can the bucket be the only durable state?
• Enroute: where should one hot repository's pushes be ordered?
• Scion (Anthropic): who should do the merging, and what sits between a laptop and the store?

Each section ends with what Loams took. The design is in design 36, with §19 for monorepo mode and §20 for the Enroute and Scion comparison. Every Loams claim carries a status from the 0.1.0 release notes: Available ships and is tested, Experimental ships but may change more, In progress is being built and does not ship in 0.1.0, and Planned is designed but not built.

Piper and CitC: a workspace is a record, not a copyCopy Anchor Link

The public account is Potvin and Levenberg, "Why Google Stores Billions of Lines of Code in a Single Repository" (CACM, 2016). Piper is the repository: one tree for most of the company, on Spanner, replicated across data centres with Paxos. As of January 2015 it held about a billion files and 35 million commits, and automated systems made more of the daily changes than people did.

Nobody clones it. Developers work in CitC (Clients in the Cloud) workspaces. A workspace stores only the files the developer changed, as an overlay on a Piper snapshot, and a FUSE filesystem shows it as the whole tree. The paper reports that an average workspace has fewer than ten modified files. Every save is a snapshot, so earlier states can be recovered, and because the workspace lives in the cloud, any machine can open it and code review and search tools can read work in progress. Development is trunk-based.

Three ideas carry over almost directly to agents:

1. A workspace costs what it changes. If a workspace is a base commit plus an overlay, creating one is a metadata write, and a million idle workspaces cost a million small records. Clones would cost a million copies.
2. Snapshots per save make work visible. A reviewer, another agent or a supervisor can read what an agent is doing before it pushes.
3. The workspace outlives the machine. An agent's sandbox can die and its successor mounts the same workspace.

What Loams took (§19, D826 to D836). A workspace is a create-only chain of snapshots under the repository's prefix, ws/<ws_id>/snap/<n>.lws. A snapshot is complete: base, HEAD, local refs, index tree, worktree tree and the workspace packs that hold new objects. Creating one is a single PUT. A save commits by creating snapshot n + 1 with a create-only PUT, the same trick the repository log uses, so one writer per workspace is enforced by the object name, with no lease service: a second mount that wants to write must write a takeover snapshot, and a fenced writer's unsaved state goes to a salvage record rather than being merged silently. Each snapshot also records a deterministic commit, which stock git can fetch as a hidden ref, refs/loams/ws/<ws_id>.

The design's own estimate on R2 list prices is about $0.03 per agent-day for an agent that saves every ten seconds for eight hours, and about $0.015 a month to keep a million idle workspaces. Those are computed, not measured.

We read three other systems for this part, and took less from each. Meta's EdenFS and Mononoke gave journal-based status, redirections that keep build output on local disk, and file sizes as derived data so that stat never fetches a blob. They are GPL-2.0, so we took ideas from public documentation only and read no code (D838). Microsoft's VFS for Git gave a warning more than a design. Its successor, Scalar, moved to partial clone, sparse checkout and fsmonitor inside git itself, because virtualisation is hard to keep correct. So Loams keeps a path with no virtual filesystem that always works: stock git with partial clone and sparse checkout. Jujutsu's "the working copy is a commit" is where the snapshot commit comes from.

Continuity: the bucket as the only durable stateCopy Anchor Link

Cursor's "Git at any scale" (Vicent Martí, 2026) describes Continuity. A write-ahead log on S3 is the source of truth, NVMe repositories on servers are warm caches, and a push is not acknowledged until it is fully persisted. Batching hides PUT latency, and rendezvous hashing picks a preferred node per repository. InfoQ's write-up reports about 120 pushes a second on S3 Standard and over 300 on S3 Express One Zone, from synthetic tests, with compaction the limit at the top.

This is the base Loams Git started from (D388 to D391), and it is still the default. Each repository has one sequencer, placed by rendezvous hashing under a lease. Every transaction that arrives while a segment PUT is in flight joins the next segment (group commit). The commit point is the create-only PUT of wal/<seq>.lgw. Correctness does not depend on there being one sequencer: two writers fence each other on the segment name, which is how git-remote-loams can push straight to the bucket while a server is also writing. Compaction runs stock git repack on a worker, never on the push path. It is the same reasoning the OLTP, OLAP, HTAP, LTAP post applies to databases: where a commit is acknowledged decides most of what a system can promise.

Enroute: the critique that one log per repository is slowCopy Anchor Link

Enroute (source, Apache-2.0, Rust) is a git-compatible server on object storage, aimed at CI, agent artifacts and embedded version control. Version 0.3.0 came out on 2026-10-14. Its article "Everyone is doing Git on S3 wrong" argues against exactly the design above. In a WAL-on-S3 design, every push to a repository contends on one log and one machine applies them in order, so a single repository's write rate is capped by that machine and by the store's PUT latency.

Enroute's answer has four parts, as its README, release notes and article describe them:

• A ledger. Each repository's refs and layer list live in a per-repository Raft log on the ledger nodes' disks ("CockroachDB-style MultiRaft"). A push is acknowledged when a majority holds the entry. Ref updates are compare-and-swap in that log, so pushes to different branches do not conflict.
• Segments and layers. Each commit becomes a segment: the objects that make up its change from its parents. Segments are grouped into immutable, pack-like layers with an index and a table of contents. A delta may only point at the same path in a parent commit, so a client that has the parent can use it directly. A fetch sends the missing commits' segments without walking trees.
• Small layers inline. Small pushes are stored in the log itself. About once a second a compute node merges recent inline layers into one file in the bucket, like an LSM memtable flush. There is no git repack.
• Stateless compute. Compute nodes keep only caches, filled by parallel ranged GETs.

Its own benchmark, forgemark, on EC2 and S3, reports a peak of 7,619 pushes a second into one repository with 512 concurrent pushers, and a 4.4 ms median push for a single pusher. Against stock git 2.56.0 it reports shallow clones more than twice as fast, partial clones on par, and full clones and incremental fetches somewhat slower. These are the vendor's numbers. We have not reproduced them. The same article lists what is not built yet, and 0.x releases do not promise to read repositories an earlier 0.x wrote.

The honest reading. On this point the critique lands. Loams' push rate for one hot repository is the group size divided by the segment PUT latency: at most 64 transactions per segment, one segment in flight. On one machine we measured 145 commits a second in memory and 93 on RustFS with 100 ms added to every PUT, with 32 pushers. We have not measured R2, S3 Standard or S3 Express One Zone at all. That meets design 36's original target of 30 pushes a second. Enroute's claim is fifty times our measured figure, on different hardware.

What Loams took. Two proposals, both Planned.

A TiKV-backed ref log (D1025). We did not want a second consensus system. Loams already runs TiKV for the metastore, Live and Loams SQL (the distributed KV post explains why), and a TiKV transaction is a Raft commit replicated to three zones in milliseconds. TikvRefLog keeps the sequencer, the state machine and the segment bytes unchanged, and moves only the commit point: one TiKV transaction that locks a tail key, writes the next log entry and the changed refs, and commits. A mirror task copies each entry into wal/<seq>.lgw in the bucket with the same sequence number and bytes, within a second as a target. Packs, checkpoints and workspaces stay on the bucket, so every bucket reader, including the helper's fetch and the restore drill, still sees the whole repository, at most that lag behind.

The cost is a real change of promise. For a repository that opts in, a push is acknowledged at the TiKV commit, before the bucket has the ref update. That amends D1 for those repositories in the same way D130 does for Live, and it is why the default stays bucket everywhere, and why tikv is refused where the deployment has no TiKV (loams dev, the desktop, every local-first backend). The planned benchmark is forgemark-style, written from Enroute's published description and not its code, against both adapters on one machine. The targets are at least 1,000 pushes a second at 512 pushers with p99 under 250 ms, and p50 under 20 ms for one pusher. Enroute's own figure is cited, not used as a target, because it ran across machines on S3.

Per-commit object locality (D1027). Enroute's segment idea, without its delta rule. Packs Loams writes, at save, push or land, are ordered as all commits, then trees grouped by the commit that introduced them, then blobs grouped the same way. A new table of contents (.lpk format version 2) records each commit's byte ranges. Reading a new commit's trees, the most common read for CI and agents, becomes one coalesced ranged GET instead of one per object on a cold cache, and --filter=blob:none reads two clusters. Packs received from stock git are stored as received, with a table of contents built during the connectivity walk the receive path already does. We did not take "deltas only against the same path in a parent", because Loams cannot write deltas yet (gitoxide has no delta compression), and stock git chooses the deltas in the packs it writes. The table of contents is an optimisation only. A pack without one falls back to index lookups.

Scion: let the server mergeCopy Anchor Link

Anthropic's "Redesigning source control at Anthropic" (2026-10-13) describes Scion, its internal version control system. It is not git-compatible and not open source, and the article says it is "by no means a suitable replacement for git for general use". It is still the closest public description of source control built for a fleet of agents. As the article describes it:

• Changesets, not packs. The unit is a changeset, the files changed relative to its first parent, pointing at compressed contents in blob storage. Spanner holds refs and metadata. Moving a ref is a compare-and-swap on one row, separate from creating changesets.
• The server does the work. Commits, rebases and merges happen on the server, so there is no push. There is no global merge-queue lock. Testing is post-merge and pre-deploy, concurrent changes can conflict, and Claude fixes the conflicts.
• Derived data such as blame, path history and commit-graph skip lists comes from one generic mechanism, computed from parent values.
• Strict tiers. A daemon per node, a regional cache that streams from memory, and an authoritative server. The tiers are strictly enforced to isolate failure domains, and each tier's hit rate is reported above 99%.
• Numbers, all Anthropic's: merges grew from about 2,500 a day in May 2026 to over 30,000 a day in September, with about 25 million CI checkouts a day, and infrastructure about 200 times smaller in CPU and memory than the git mirrors it replaced.

Two open projects followed the article. CarterT27/scion (Apache-2.0) calls itself an independent clean-room implementation, uses CockroachDB and S3-compatible storage, and puts git compatibility out of scope. openscion follows Scion's design and is GPL-3.0-only. We take nothing from it and do not read its source. Scion's ideas reach Loams only through Anthropic's public article.

Loams already had the server side of this. Workspaces have ApplyEdits, Rebase and a server-side Diff for agents that never mount anything, plus Log, Blame, Grep and DiffStat over derived data (D833, D834). Those are merged on dev (In progress: built, not served by the release images). PushWorkspace builds on the server the closure of objects the repository lacks from the workspace's own packs, so a push moves no data from the client. It is in review (In progress). What was missing was the last step Scion makes routine.

Land (D1026), Planned. One call that rebases a workspace's stack of commits onto the target branch on the server, then compare-and-swaps the ref through the repository's sequencer, and retries when another land wins.

A conflict returns a structured result (path, kind, and the three versions) and writes nothing, so the agent fixes it in its workspace and lands again. An optional pre-land checks hook exists, off by default, because Scion's post-merge testing is the better default for agent volume. A retry with the same land id gets the first receipt, so a lost acknowledgement never lands twice. There is no merge queue. A lost race costs a re-merge of the paths the new tip changed. Whether a branch should be able to require Land and refuse direct pushes is open (Q907; the proposed default is a land_only flag, off).

The positioning that came out of the comparison is one sentence in D1029: Loams Git keeps full git compatibility while offering Scion-style server-side workspaces, history and landing. Scion chose to leave git behind. We think most teams cannot, and that agents benefit from the same interface every CI system and person already uses.

A regional cache tier (D1028), Planned. loams-git-cache is Scion's regional cache for Loams. It is a stateless read-through cache in a region or site, between per-host loams-vfsd daemons or CI nodes and the server. It speaks the same RPCs, so a client points at it with no other change. It caches immutable, content-addressed data only: objects by id, tree sizes by tree id, blame by commit and path, pack ranges by checksum. Anything mutable or ref-relative goes through. Tiers are strict, as in Scion: a client talks only to its configured cache, a cache only to the server, and neither skips a tier when the next one fails, so a cache outage cannot dump a region's read load on the server. Access is decided by the server, cached for 60 seconds per token, repository and workspace, and entries are partitioned by namespace and fork family, so nothing is shared across tenants.

Licences: ideas, and where they came fromCopy Anchor Link

Half of what we read is under licences we cannot take code from, so D838 and D1029 set the rules in writing:

Every pull request that implements a §20 idea cites the public document it worked from.

What ships, and what does notCopy Anchor Link

In 0.1.0, git-remote-loams Experimental is the way in. With bucket credentials, a laptop or CI runner can clone, fetch and push with nothing else running, and with LOAMS_GIT_CREATE_ON_PUSH=1 the first push creates the repository. The protocol post has the details and the limits: no Smart HTTP, no partial clone, no --force-with-lease.

On dev, GT1 has moved past the core. The repository catalog, sequencer placement and forwarding, the _git event mirror, range reads from stored packs, tree sizes, the commit graph with Log and DiffStat, Blame and Grep, the workspace snapshot format and store, workspace packs and mount-less editing are merged. They are behind the git cargo feature, which the release images do not build, so none of it is served in 0.1.0. PushWorkspace is in review. Smart HTTP, the server's receive-pack, the FUSE client and the shim are next in the plan. The five §20 proposals are GT1 Tasks 88 to 92.

The trade-offsCopy Anchor Link

• The TiKV adapter weakens a promise. "Acknowledged means it is in the bucket" is simple. For opted-in repositories it becomes "acknowledged means it is in a three-zone Raft log, and in the bucket within about a second". Q906 asks whether that is acceptable. The alternative, waiting for the mirror too, brings back the PUT latency we were trying to remove.
• Group commit still shares one sequence. Pushes to different branches still serialise on the tail key. Per-ref transactions would remove it, and we will not build them unless the benchmark shows tail is the limit.
• Workspaces are a second product to keep correct. VFS for Git's retirement is the warning. The path with no virtual filesystem, stock git with partial clone, has to keep working on its own.
• Our numbers are small and local. 93 pushes a second on one machine says little about R2 or S3. Until the benchmark runs against a cloud store, the comparison with Enroute is a comparison of a measurement with a claim.

What we take from all thisCopy Anchor Link

The four systems agree on more than their designs suggest. Piper and Scion both moved the expensive work to the server and made the client thin. Continuity and Enroute both made the bucket the place data lives and argued only about where order is decided. Scion and Enroute both reject the idea that one machine should apply every push to a hot repository. What none of them offers is the combination: stock git on the wire, Scion-style workspaces and landing on the server, and a bucket you own as the only state you have to back up. That is what Loams Git is for. Part of it ships today as a remote helper; the rest is on dev or written down in design 36, where you can check our reasoning.

The next post in this series turns from code to agent memory: what HydraDB, Mem0, Graphiti and Zep taught us about keeping memory as data.