Loams Git

Loams Git stores git repositories in object storage. A repository is a prefix in the bucket that holds a write-ahead log of create-only segments, periodic checkpoints and immutable, content-named packs. The bucket is the source of truth: any node, or any client that holds bucket credentials, can rebuild a repository's state from it. Forks cost one small object. Stock git reaches a repository through git-remote-loams, a remote helper that talks to the bucket directly, with no Loams server in between.

It is built for agent fleets, not for people browsing code. Thousands of small repositories, agents that each want their own branch or fork, and pushes that must be durable the moment they are acknowledged. The direction is a Google Piper and CitC-style monorepo on the bucket: server-side workspaces that cost one object each, a filesystem that fetches only the files you touch, and a git wrapper that takes over the common commands. Only the storage core, the repository service and the helper exist today.

Use it to experiment with git on a bucket you control: back up or share repositories without running a git server, give each agent a cheap fork, or build on the WAL and its event stream. Do not use it as your team's forge. It is not one, by design: no pull requests, no issues, no CI, no web UI. It is experimental in 0.1.0: its on-disk formats can still change, and the server half is not in the release images.

What shipsCopy Anchor Link

The 0.1.0 feature table lists Loams Git as Experimental: "git-remote-loams ships with the loams CLI packages and pushes and fetches straight to a bucket; the repository service is not in the release images". The helper is the first piece to ship; the two previews held Loams Git back entirely, because it had no push or fetch path yet. The console's Git page still shows demonstration data.

ArchitectureCopy Anchor Link

Every repository lives under one prefix (design 36 §4.1):

ns/<ns>/repos/<repo_id>/
  head                          hint: last seq and checkpoint; best effort, at most one write a second
  wal/<seq:020>.lgw             create-only segments; segment n's PUT commits its batch
  checkpoints/<seq:020>.lgc     create-only; the full state after segment seq
  packs/<checksum>.lpk          create-only, content-named: pack, index and footer in one object

Repositories have names through a catalog: _names/<name>.json is written after the repository's checkpoint 0 and is the commit point of a create or a fork. Deleting writes a _deleting/ marker and a purge removes the objects after the grace period, unless a fork still reads them.

How it worksCopy Anchor Link

The commit point is a create-only PUTCopy Anchor Link

A writer that holds the state after segment n commits a batch of ref transactions by writing segment n+1 with If-None-Match: * (D389):

• 200: committed at n+1.
• 412: someone else owns n+1. Read it, apply it, re-check this batch's expectations against the new state, try n+2.
• Timeout, 5xx or 429: the outcome is unknown. Read wal/<n+1> back. If it carries this batch's random batch_id, the batch committed; if another batch's, treat it as a 412; if it is missing, resend the same bytes.

Exactly one batch can own a sequence number on every store Loams supports: S3, R2, GCS (ifGenerationMatch=0), Azure and RustFS all refuse a second create of the same name. There is no separately compare-and-swapped head pointer: it would cost a second PUT per batch, add a second point of truth, and contend on one key, and R2 allows one write per second to a key. Readers read forward from a checkpoint and stop at the first missing number, so "segment n+1 is 404" proves n is the tail; a linearizable read is one GET past the state a node already holds. This is the same trick as Delta Lake's transaction log, where a sequential, create-only file name is the compare-and-swap. See The bucket is the source of truth and The log is the spine for the general pattern in Loams.

FormatsCopy Anchor Link

A segment (.lgw, format version 1) is a header with magic LGITWAL\0, the sequence number and the batch id, a body that is a batch of CloudEvents 1.0 in the protobuf format, and a CRC32C trailer. Segments are capped at 1 MiB on every store, which also keeps them inside RustFS's atomic conditional-PUT size. Each event is one transaction: its id is the transaction's idempotency key, its type is io.loams.dev.git.reftxn.v1 (or packset.v1 and config.v1), and its data is a loams.git.v1 message. A push stores its pack as one object, packs/<checksum>.lpk, named by the pack's trailer checksum, so a retried upload writes the same name and either racing creator is correct. Readers accept format versions N and N−1. Golden files for every format are checked into crates/loams-git/tests/golden/.

One sequencer per repository, with group commitCopy Anchor Link

The engine runs one sequencer task per active repository, on the rendezvous owner of the repository, under a lease (D391). It holds the current state and a queue of validated transactions. One segment PUT is in flight at a time; transactions that arrive meanwhile form the next group (at most 64 transactions or 1 MiB), so groups grow with load and no linger is needed. Inside a group each transaction is checked against the state left by the ones before it: idempotency, each ref's expected old value, protections (deny_delete, deny_force) and the writer's scope. A transaction that fails is answered on its own and dropped from the group.

Correctness never depends on there being one sequencer. Two writers on one repository fence each other on the segment name: the loser reads the winner's segment, re-validates, and retries. That is exactly what lets the helper work with no server: each git push process is its own sequencer for the length of the push.

IdempotencyCopy Anchor Link

A git push's key is derived from the principal, the pack checksum and the sorted (ref, expected, new) triples. Keys are kept for one hour in the sequencer's index and in every checkpoint, together with a digest of the request, so a retry after a restart or a fence is answered with the original receipt, and a key reused with a different request is refused. After the window, the old-value expectations still decide: a replayed fast-forward finds the ref already moved and is rejected, never applied twice. The helper's test repeated_push_after_lost_answer_is_replayed covers the lost-answer case.

Checkpoints, forks and GCCopy Anchor Link

A checkpoint holds the full ref state, the live pack set, the fork parent and the idempotency window. The sequencer writes one every 256 segments, or when replay since the last one passes 8 MiB, off the commit path.

A fork is O(1). Forking repository A at sequence s writes B's checkpoints/00000000000000000000.lgc with parent = A at s, A's refs at s and an empty pack set: one create-only PUT, no pack or segment copied. Object lookups in B fall through to A's packs as of s, recursively, up to 8 parents. B then has its own WAL; A carries on independently.

GC is careful because readers trust the hint and forks read their parents. Segment GC keeps everything from the newest checkpoint that Exactly(seq) reads may still need (24 hours), the checkpoint the hint names, and the checkpoint at or below the oldest live fork point; it deletes older segments and checkpoints, and segments above a gap, only after the grace period, and refuses a plan if the hint moved below it meanwhile. Pack GC by fork-family reachability is designed (D396) and not built yet.

Reads and the event streamCopy Anchor Link

A push returns its sequence number as a consistency token, git:<repo_id>:<seq>, so an agent that pushes and then reads can ask for at least that state (compare consistency tokens in the engine). After a segment commits, a mirror task publishes each transaction's CloudEvent to the reserved namespace stream _git (8 partitions, keyed by repository id). Public producers are refused on _git, so its events cannot be forged, and an hourly repair publishes whatever the WAL holds past the mirror's cursor. Code indexing and usage consumers read it.

A push through the helperCopy Anchor Link

git runs git-remote-<scheme> for a <scheme>::<address> URL, so with git-remote-loams on PATH, loams::<store-url>/ns/<ns>/repos/<repo> works with stock git. The store URL is any object_store URL (file://, s3://, gs://, az://) with credentials from the usual environment (AWS_*, GOOGLE_*, AZURE_*). <repo> is a repository id, or a name the catalog maps to one.

• list and fetch read the latest checkpoint and the segments after it, and write nothing, so read-only credentials can clone. Fetch downloads each live pack the local repository lacks, fork parents' included, in 16 MiB range reads, checks each pack's trailer against its name, and installs it with its stored index.
• push checks fast-forwards locally, runs git pack-objects --revs --thin into git index-pack --fix-thin once for all refspecs, uploads one .lpk, and commits one ref transaction (or one per ref, unless --atomic). A commit answered "unavailable" is repeated with the same key, at most five times.
• Limits: 4,096 refs per push; a push to a missing repository fails unless LOAMS_GIT_CREATE_ON_PUSH=1, which creates it with HEAD on the first branch pushed.

The helper's tests drive the real git binary: a full round trip, non-fast-forward rejected with "fetch first", forced pushes, deletes, atomic multi-ref pushes, push options recorded in the WAL event, clone of a fork, eight concurrent git push processes on distinct branches and on one branch (exactly one wins per round), "never lose an acknowledged push", and a helper racing a server-side sequencer on the same repository.

Key decisionsCopy Anchor Link

Full entries are on the decisions page; the design is design 36, which extends §3 of design 15.

• D388: a per-repository WAL instead of one refs document. Design 15 kept a repository's refs in a single compare-and-swapped document. Rewriting it on every push costs O(refs) and contends on one key, and agent fleets create thousands of branches.
• D389: the commit point is the create-only PUT of the next segment. No head pointer to CAS, one fewer PUT per batch, and no collision with R2's one-write-per-second-per-key limit. Delta Lake's log uses the same protocol.
• D390: CloudEvents in segments, one object per pack. Segments are standard events with an idempotency id, capped at 1 MiB; a pack, its index and a footer are one content-named object, so a push costs one PUT for its data.
• D391: one sequencer per repository, with group commit, but correctness never depends on there being only one. Taken from Cursor's Continuity design (an S3 WAL as the truth, acknowledge only after persisting, batching), without its gossip and replica fleet.
• D395: git-remote-loams is serverless first. The first phase needs no server; stateless-connect and loams:// addresses come with Smart HTTP.
• D394 and D396: Loams' own server loop on gitoxide, stock git as a process. gitoxide's server-side plumbing, delta compression and bitmap writing are not finished, so serving uses Loams' loop over gitoxide primitives, and repacking will run unmodified git repack as a separate process. git is GPL-2.0 and is never linked.
• D825 to D838 (proposed): monorepo mode. The owner's direction of 2026-10-10: Piper and CitC-like on the bucket, download only touched files, millions of workspaces, a git wrapper that takes over. These decisions are proposed, pending the owner's confirmation. D838 keeps it clean-room: no code from Sapling, EdenFS, Mononoke or git is copied or linked.
• D1025 (approved): an opt-in TiKV ref log for hot repositories. One repository's push rate on the bucket WAL is bounded by group size over segment PUT latency (the log records 93 to 145 commits per second measured at 100 ms per PUT; cloud stores are not measured yet). A TiKV commit is milliseconds, so hot repositories may move their commit point to TiKV while the bucket keeps a complete mirror. The bucket stays the default. Planned as GT1 Tasks 88 and 89.
• D1029: positioning. "Loams Git keeps full git compatibility while offering Scion-style server-side workspaces, history and landing."

What is compatibleCopy Anchor Link

• git. Stock git with git-remote-loams on PATH. The helper runs git pack-objects and git index-pack as processes, so git must be installed. Its tests run against the git on the CI runner; the plan's Task 0 probes used git 2.55.0. No minimum git version is pinned.
• Object format. SHA-1 only; the object format is a field in every format version, and SHA-256 repositories are a later step.
• Stores. Any object_store URL with conditional create: S3, R2, GCS, Azure and RustFS, plus local file:// paths. The helper's tests run on local directories.
• Not yet. Smart HTTP, so libgit2, JGit and hosted CI systems that only speak HTTPS cannot reach a repository. --force-with-lease (the helper answers option cas as unsupported). Pushes larger than memory (the helper reads its pack whole).

Compared with the alternativesCopy Anchor Link

GitHub and GitLab are where people collaborate on code, and Loams Git does not try to replace them: no pull request UI, issues or CI is a stated non-goal. Keep your forge; Loams Git is for the storage behind agents, where a repository per agent or a fork per task has to be cheap and the bucket has to be the truth.

Plain git-on-S3 helpers, such as awslabs/git-remote-s3, are the closest thing in shape: a bucket as a serverless remote. They are simpler and already usable. Loams Git's helper adds a write-ahead log with a create-only commit point, fencing between concurrent pushers, replay of a push whose answer was lost, checkpoints, O(1) forks, a catalog and GC, and the same repository is also served by the engine. The cost is that the bucket holds Loams' own formats (.lgw, .lgc, .lpk), not a bare repository other tools can read without the helper.

Google's Piper and CitC are the inspiration for where this is going, from Potvin and Levenberg's 2016 CACM paper: one central repository, and CitC workspaces in the cloud that store only the files a developer changed, as an overlay on a repository snapshot, shown through FUSE, with every save a snapshot. Piper is not git, and it is Google's internal system. Loams Git takes the ideas and keeps git compatibility: workspaces as server-side records are built, and the lazy filesystem and the git wrapper that would make it feel like CitC are planned.

What is missing and what is plannedCopy Anchor Link

From the GT1 plan, which supersedes the earlier GT1, GT2 and GT3 plans of 2026-10-01 and is In progress:

• Interop (Tasks 46 to 52). Smart HTTP with protocol v2 upload-pack (ls-refs, fetch with filters and shallow, object-info) and v0/v1 receive-pack; stateless-connect and loams:// addresses in the helper for partial clone and lazy fetch; a client matrix of git, gitoxide, libgit2 and JGit.
• Compaction and GC (Tasks 53 to 55). Geometric repack with bitmaps and a multi-pack index by stock git repack, committed through the WAL like a push; pack GC across fork families; integrity scrubs.
• Monorepo mode (Tasks 27 to 45). PushWorkspace, loams-vfs (a lazy FUSE filesystem on Linux) and the git shim that runs common commands natively and falls through to real git with a one-line notice.
• Secrets as code (Tasks 59 to 67). env/ files whose values never become git objects.
• Build cache and crates mirror (Tasks 68 to 73). An sccache backend and a crates.io sparse-index read-through. The buildcache and registry cargo features are reserved; their crates do not exist yet.
• Production (Tasks 74 to 82) and the follow-ups approved on 2026-10-16: the TiKV ref log, Land (a server-side rebase and land), per-commit object locality and a regional cache tier (Tasks 88 to 92).

Try itCopy Anchor Link

From docs/release/packaging.md: every 0.1.0 package installs git-remote-loams next to loams. With it on PATH:

git-remote-loams --version
git clone loams::<store-url>/ns/<ns>/repos/<repo> work
git -C work push origin main

To create a repository, push to an address that does not exist yet with LOAMS_GIT_CREATE_ON_PUSH=1. With a local directory as the store:

LOAMS_GIT_CREATE_ON_PUSH=1 git push loams::file:///srv/loams/ns/1/repos/web main
git clone loams::file:///srv/loams/ns/1/repos/web web-copy

From source, the helper builds on its own, without the server's features:

cargo build -p loams-git-remote

The server half (loams.repos.v1) needs a loams built with --features git. It is not in the 0.1.0 images.

SourceCopy Anchor Link

• Crates: crates/loams-git, crates/loams-git-remote
• Design: 36, Loams Git, 15, Agent workspaces
• Plans: GT1, Loams Git in production; superseded: GT1 WAL git core, GT2 Smart HTTP, GT3 build cache and mirror
• Packaging: docs/release/packaging.md