On this page
Loams Git keeps repositories in the bucket: a write-ahead log of create-only segments, content-named packs, checkpoints and O(1) forks. Storing them is half the job. The other half is that people and agents use git, unmodified, and expect clone, fetch and push to work. Git gives two ways in. A server can speak the smart protocol over HTTP, or the client can run a remote helper, a program git drives over stdin and stdout for URLs of the form <transport>::<address>. Loams ships the second today and is building the first.
| Specs | gitprotocol-v2, gitprotocol-pack, gitprotocol-http, gitprotocol-common, gitformat-pack, gitremote-helpers |
| Crates | loams-git (WAL, ref log, packs, catalog), loams-git-remote (the git-remote-loams binary), the git listener in loams |
| Design | §36 Loams Git; D388–D399; monorepo mode D825–D847 (proposed) |
| Tested with | Stock git driven end to end in CI (gt1.yml, GitHub's ubuntu-24.04 runner, which prints its git version in the job), with RustFS 1.0.0 as the bucket |
| Status | Experimental git-remote-loams (in the Loams 0.1.0 CLI packages) · Planned Smart HTTP · Planned monorepo mode |
What the protocol is
Framing: pkt-lines
Everything git says on the wire, except the packfile bytes themselves, is a pkt-line: four lowercase hex digits giving the length of the line including those four digits, then the payload. 0014command=ls-refs\n is twenty bytes. A pkt-line carries at most 65516 bytes of data. Three lengths are reserved as control packets: 0000 (flush, the end of a message), and in protocol v2 0001 (delim, between sections of a request) and 0002 (response-end, used by stateless transports).
Once the pack starts flowing, a side-band multiplexes it: each pkt-line's first byte names a band, 1 for pack data, 2 for progress text (what you see as remote: Counting objects…), 3 for a fatal error. The packfile itself is a separate format: the signature PACK, a version (2), an object count, then each object zlib-compressed, either whole or as a delta against another object, by offset in the same pack (OFS_DELTA) or by object id (REF_DELTA), and a trailing hash of everything before it. A thin pack contains deltas against objects it does not include, on the assumption that the receiver has them; the receiver must "fix" it before it can be indexed.
The handshake: v0 and v2
In the original protocol (v0, and v1 which only adds a version line), the server talks first. upload-pack (the fetch side) and receive-pack (the push side) open with a ref advertisement: every ref and its object id, with the server's capabilities after a NUL on the first line. On a repository with a hundred thousand refs, a git fetch of one branch still downloads all hundred thousand lines.
Protocol v2 fixes that by making the server advertise capabilities, not refs, and by turning the conversation into commands. The client asks for v2 (over HTTP with the header Git-Protocol: version=2, over SSH with the GIT_PROTOCOL environment variable) and the server answers with lines such as version 2, agent=…, ls-refs=unborn, fetch=shallow filter and object-format=sha1. Then the client sends requests of the form command=<name>, capabilities, a delim-pkt, arguments and a flush:
ls-refslists refs, optionally only those with a givenref-prefix. A fetch ofmainasks only aboutrefs/heads/main.fetchnegotiates: the client sendswant <oid>lines andhave <oid>lines for what it already has, the server acknowledges common commits, and once both sides agree the server sends apackfilesection on the side-band. Arguments add shallow clones (deepen), partial clone (filter blob:none), andwait-for-done.object-inforeturns object sizes without the objects.
Protocol v2 has no push. A push still uses receive-pack in v0/v1 form: the server advertises refs and capabilities such as report-status, atomic, delete-refs, side-band-64k, ofs-delta and push-options; the client sends one <old-oid> <new-oid> <ref> command per ref, a flush, and the pack; the server answers with ok <ref> or ng <ref> <reason> per ref.
Smart HTTP and authentication
Over HTTP the same exchange becomes request and response. The client starts with GET <repo>/info/refs?service=git-upload-pack (or git-receive-pack), and then POSTs to <repo>/git-upload-pack or <repo>/git-receive-pack with content types such as application/x-git-upload-pack-request. Each POST is self-contained, which is why v2 needs the response-end packet: the server cannot keep state between requests. Authentication is plain HTTP, normally Basic with a token as the password, or a Bearer token; git asks its credential helpers when a server answers 401 with a WWW-Authenticate challenge.
Remote helpers
A remote helper inverts the arrangement. For a URL loams::<address>, git runs git-remote-loams <remote-name> <address> and speaks a line protocol on the helper's stdin and stdout. It asks capabilities, and the helper names what it supports, from a list that includes fetch, push, option, import, export, connect and stateless-connect. Then git sends list (or list for-push) and expects ref lines back, followed by batches of fetch <oid> <name> or push <src>:<dst> lines, each batch ended by a blank line, with ok <dst> or error <dst> <why> per pushed ref. option lines pass flags such as verbosity, progress and --atomic. connect and stateless-connect let a helper tunnel the smart protocol instead of implementing fetch and push itself; the documentation still marks stateless-connect as experimental and for internal use.
Why Loams speaks it
Git is the interface every coding agent, CI system and developer already uses, so Loams Git is useless if it needs a different client. The bucket is where the repository lives (D388) because the same reasons that put the engine's data there apply to code: one durable copy, compute that can come and go, forks that cost a pointer, and millions of agent workspaces that would be absurd as millions of clones.
We chose two access paths on purpose (D394, D395):
- A remote helper that needs no server.
git-remote-loamsreads and writes the bucket directly. A laptop or CI runner with bucket credentials can clone and push with nothing else running. It is also the cheapest way to prove the storage format against real git. - Smart HTTP on the server, for clients that cannot hold bucket credentials, for agents in sandboxes, and for partial clone and lazy fetch at scale. Here
upload-packspeaks protocol v2 andreceive-packv0/v1, since v2 has no push.
Stock git runs only as an unmodified separate process: as the test oracle, as the repack worker, and inside the helper for the pack steps of a push. It is never linked. The server loop is ours, on gitoxide primitives, because gitoxide's own server-side upload-pack and receive-pack plumbing is not finished and because the serving path must run on nodes with no local repository.
How Loams implements it
git-remote-loams today
The helper lives in crates/loams-git-remote, and the Loams 0.1.0 CLI packages install it next to loams (/usr/bin on Linux, the macOS tarballs, the Windows zip). It is not published to crates.io. Addresses name a bucket URL and a repository:
git clone loams::s3://my-bucket/prefix/ns/1/repos/web work
git clone loams::file:///srv/loams/ns/1/repos/web work
git -C work push origin mainEverything before /ns/<ns>/repos/<repo> is an object_store URL (s3://, gs://, az://, file://), opened with that backend's credentials from the environment (AWS_* and friends). The last segment is a repository id, or a name the repository catalog maps to one.
It answers capabilities with fetch, push and option. The options it accepts are verbosity, progress, atomic, push-option and dry-run; anything else is answered unsupported.
listreads the WAL's tail, the latest checkpoint plus the segments after it, and prints the refs andHEAD. It writes nothing, so read-only credentials can clone and fetch.fetchdownloads every live pack the local repository lacks, including a fork parent's packs up to its fork point, in 16 MiB range reads. It checks each pack's trailer against its name and installs the.packand its stored.idxinto.git/objects/pack/.pushchecks fast-forwards locally, runsgit pack-objects --revs --thinintogit index-pack --fix-thinonce for all refspecs, uploads onepacks/<checksum>.lpk(pack, index and a footer in one object), and commits the ref updates.
The commit is the interesting part. The helper opens its own ref log for the length of the push and acts as its own sequencer: it commits by creating the next WAL segment with a create-only PUT, as described in the S3 post. If another helper or a server got there first, the PUT fails, the helper reads the winner's segment, re-validates and tries the next number. Any number of helpers and servers can write one repository; the bucket decides the order. A push with --atomic is one ref transaction; without it, one transaction per ref, so each ref gets its own ok or error.
Pushes are idempotent. The key is a hash over the pusher's principal (the S3 access key id, the Azure storage account, or local:<user> for a directory) and a digest of the pack checksum and the sorted ref updates. A commit whose answer was lost is retried with the same key, up to five times, and replayed rather than applied twice.
The CI suite drives real git through clone, fetch and push round trips, non-fast-forward rejection, forced pushes, ref deletion, atomic multi-ref pushes, push options, forks, concurrent pushers to one branch and to distinct branches, a helper racing the server on one repository, and lost acknowledgements injected into the store.
The git listener and Smart HTTP
The loams server has a git listener (--git-listen, loopback only until the unified auth work lands) that answers /git/<ns>/<repo>.git/…. Authentication is already real: a token file of SHA-256 hashes (--git-tokens), each mapped to a grant in one namespace, presented as the HTTP Basic password or a Bearer token. The namespace comes from the token, never from the URL: a URL naming another namespace, a repository the token does not cover, and a repository that does not exist all answer the same 404. A wrong token gets 401 with Basic realm="Loams Git", and no token ever reaches a log line or a response.
Smart HTTP itself is not served yet. An admitted request for an existing repository answers 501, with a message pointing at loams:: URLs. The design is fixed (§36 §6.1): a v2 advertisement with ls-refs=unborn, fetch=shallow wait-for-done filter, server-option, object-format=sha1 and object-info; packs assembled from stored entries, reusing deltas whose base is also sent, and whole stored packs streamed when the wants cover them; and a v0 receive-pack advertising report-status report-status-v2 delete-refs side-band-64k quiet atomic ofs-delta push-options, whose receive path streams the pack into a spill file, indexes and verifies it, writes one .lpk, and answers report-status only after the segment commits, with the sequence number on side-band 2.
Monorepo mode
The larger direction, proposed in §36 §19, is Piper and CitC on a bucket: workspaces as server-side records (a base commit plus an overlay, saved as a create-only chain of snapshots), a lazy FUSE client that fetches files on first read, and a git shim that runs common porcelain natively and falls through to real git for anything else. Stock git stays the interop path through Smart HTTP, partial clone and sparse checkout, and each workspace is visible to it as a hidden ref. The workspace store is being built in loams-git; the FUSE client and the shim are not.
What is not supported
- Smart HTTP, and with it any client that has no bucket credentials. The design covers HTTP only;
git://and SSH transports are not in it. stateless-connectandloams://<host>/<ns>/<repo>addresses in the helper, and with them partial clone (--filter=blob:none) and lazy fetch.--force-with-lease:option casis answeredunsupported.- Pushes larger than memory through the helper, which reads its pack whole. The server's receive path is designed to stream.
- SHA-256 repositories over the wire: the log's format can record either hash, but the planned advertisement is
object-format=sha1only. - The Loams Git repository service in the release container images.
Gotchas
loams:: is not a scheme, it is a transport prefix. git looks for a helper named git-remote-loams on PATH for any URL that starts with loams::. If the helper is not on PATH (the Windows zip and macOS tarballs need it added), git reports that it cannot find the remote helper, not that the repository is missing.
The helper needs git installed. It runs pack-objects, index-pack, merge-base and rev-parse as processes.
A push to a missing repository fails. That is deliberate: a typo should not create a repository. Set LOAMS_GIT_CREATE_ON_PUSH=1 for the first push, which writes checkpoint 0 with HEAD on the first branch pushed.
Fetch is pack-granular. The helper downloads whole stored packs, not a negotiated pack, so a fetch after many small pushes downloads many small packs until the repository is repacked (D396). Negotiation is what Smart HTTP's fetch adds.
deny_force does not refuse a forced fast-forward. +main that happens to be a fast-forward is an ordinary update. Existing tags never move without force.
Protocol v2 on HTTP depends on a header. Proxies that strip unknown headers silently drop Git-Protocol, and the client falls back to v0 and its full ref advertisement. When Smart HTTP lands behind a proxy, check that the header survives.
Thin packs need their bases. A thin pack is only valid against the receiver's object set. The helper fixes thin packs locally before upload, and the planned receive path resolves bases from the pack set at the snapshot its advertisement came from, refusing a pack that relies on one compaction has since retired.
The next post moves from git to agents: the Model Context Protocol, and how loams-agentd and Studio speak it.