The bucket is the source of truth

The Loams engine keeps every durable byte in one object storage bucket: S3, GCS, Azure Blob, or an S3-compatible store such as RustFS or MinIO. Nodes keep caches and indexes they can rebuild, and a small metastore keeps pointers and offsets. If you deleted every node and kept the bucket, you would lose no acknowledged data.

This is the first design principle in the engine's architecture, and it was the first decision written down (D1).

Why we built it this wayCopy Anchor Link

A typical AI application stack runs a search cluster, a vector database, a graph database, a log and an analytics store. Each one replicates its data two or three times on block storage, is sized for peak load, and runs all day. Each one is a separate copy of the same entities, kept in step by connectors that drift.

Object storage changes the arithmetic. It is already replicated across zones by the provider, it costs a fraction of replicated block storage per byte, and it has no servers for you to size. Since 2024, S3 also supports conditional writes (If-None-Match, If-Match), and GCS and Azure have equivalents. That is the missing primitive: with create-only writes and compare-and-swap, a database can commit to a bucket safely without a lock service in front of it.

So our opinion is simple. Durability is the bucket's job, and the bucket does it better than a fleet of disks we would have to operate. Everything Loams runs on a machine should be something it can lose.

How it worksCopy Anchor Link

Objects are immutable. Every data object is written once, under a name that includes a ULID or a version number, with a create-only conditional PUT. Nothing is updated in place. A WAL object, a log segment, a Lance fragment, a Tantivy split, a delete bitmap, a collection manifest: each is written in full and then only read.

Only pointers move. To change what a collection contains, a worker writes new objects and then moves one pointer in the metastore, from manifest v to manifest v+1, with a compare-and-swap. Readers that loaded v keep a consistent view; new readers see v+1. See collections and manifests.

The metastore holds metadata, never object data. Streams assign offsets at a rate that conditional PUTs on a bucket cannot sustain (tens to hundreds of milliseconds each, contended per key), so offsets, leases and pointers live in a metastore: an embedded Raft group by default, or TiKV. Its snapshots go to the bucket too. See metadata and TiKV.

Caches never need invalidation. Because an object never changes after it is written, a cached byte range of it is valid forever. A node only has to learn that a pointer moved, which the metastore tells it through a watch.

Garbage collection is by reachability. An object is deleted only when no live manifest, retained manifest, offset index entry or tag refers to it, and it is older than a grace period (one hour by default, longer than any query). Time travel falls out of this: a superseded manifest stays readable for 24 hours by default.

The layout is plain and inspectable. A namespace's objects live under ns/<namespace_id>/, and you can list them with any S3 tool:

s3://<bucket>/<cluster_prefix>/
  wal/<class>/<node_id>/<ulid>.wal              # WAL objects, many namespaces each
  ns/<namespace_id>/
    streams/<stream_id>/<partition>/….seg       # log segments
    collections/<collection_id>/
      lance/…                                   # Lance dataset
      text/splits/<ulid>.split                  # Tantivy splits
      text/deletes/<split_ulid>/<ulid>.bitmap   # delete bitmaps
      manifests/<version>-<ulid>.pb             # immutable manifests
      hot/hnsw/…                                # derived HNSW artifacts
    pk/<object_id>/…                            # primary-key index (SlateDB)

What we give upCopy Anchor Link

• Write latency. A write is acknowledged after a WAL object PUT and a metastore commit. On regional S3 that is hundreds of milliseconds at the tail, not single digits. Writers batch: a log node buffers many partitions for up to 250 ms or 8 MiB before one PUT. The design adds two lower-latency classes for streams that need them (express on zonal buckets, quorum on a Raft journal), and neither is built yet.
• Cold reads. A query against data no node has cached pays object storage round trips. The design target is half a second to a second at the median for a cold collection query, against 5 to 20 ms warm. The hot tier exists to make the common case warm.
• Request costs. Object stores charge per request. Batching writes, coalescing range reads and caching footers is not optional; it is part of the design.
• No interactive transactions in the engine. The retrieval engine does not offer millisecond, multi-statement transactions over mutable rows. That is a deliberate boundary, not an oversight: the Loams products that do need them (Loams Postgres, Loams SQL, Loams Live) keep their commit path off the bucket. See OLTP, OLAP and HTAP.

How other systems do itCopy Anchor Link

• turbopuffer serves search and vectors from object storage with a cache on NVMe. It is the closest design to the Loams engine, and it is closed source.
• WarpStream and AutoMQ put Kafka's log on object storage. WarpStream's leaderless, write-to-bucket path is the model for Loams' standard WAL class. AutoMQ's open-source edition writes its WAL to S3 too; its low-latency WAL options are commercial.
• Neon keeps Postgres pages on object storage but acknowledges commits on a quorum of safekeepers first, because a database commit cannot wait for S3. Loams Postgres follows that model.
• Elasticsearch, Qdrant and Neo4j keep their primary copy on local or attached disks and replicate it between nodes. Losing a node means re-replicating data; adding one means copying it.
• StarRocks in shared-data mode and ClickHouse Cloud separate compute from storage the same way for analytics, with a RAM and NVMe cache in front of the bucket.

StatusCopy Anchor Link

Available The engine's collections, streams, links, manifests and garbage collection keep all their data on the bucket in Loams 0.1.0. The standard WAL class is built; express and quorum are designed but not built.

Loams Live and Loams SQL keep their data in TiKV, which replicates with Raft on local disks. That is a stated exception for those products (D130), not for the engine.

Go deeperCopy Anchor Link

• The log is the spine, the next idea in the chain.
• Stateless compute: what this principle makes possible for nodes.
• Loams Engine, the product this page describes.
• Design: 01 Architecture (§1 and §6), 03 Storage formats (§7, garbage collection), decision log (D1, D25, D27).
• Source: crates/loams-store (object store access), crates/loams-log (WAL objects and segments).