How Loams works

Loams is an open-source data and execution platform for AI applications, built in Rust. Search, live data, durable workflows, graphs, analytics and SQL share one design: your bucket holds every durable byte, an ordered log carries every change, and compute holds nothing it cannot rebuild. This page walks through that design once, end to end. Every idea links to a page that explains it properly.

Three ideas carry everythingCopy Anchor Link

1. The bucket is the source of truth. Documents, vectors, indexes, tables and the log itself live in object storage you own, in open formats. A cold namespace costs only its bytes. Read more.
2. The log is the spine. Every write is appended to an ordered log before anything else happens. Collections, tables and graphs are materializations of that log, so they can always be rebuilt from it. Read more.
3. Compute is stateless. A node holds caches and in-flight work, nothing more. Nodes start fast, scale out freely and can fail without losing data. Read more.

A write, end to endCopy Anchor Link

When your application writes a document, the request reaches a gateway, which appends it to the log. Once the log entry is durable, the write is acknowledged with a consistency token. In the background, a link applies the record to its collection and publishes a new manifest, which ties a Lance dataset version to its Tantivy text-index splits in the bucket.

Read the write path, collections and manifests and consistency tokens.

A read, end to endCopy Anchor Link

A query names what it wants: a vector, keywords, sparse terms, filters, or any mix of them. The planner runs each retrieval signal over the current manifest and fuses the results into one ranking (hybrid retrieval). If the query carries a consistency token, the read also merges the log's not-yet-indexed tail, so it sees every write that token covers.

The data itself is served from the fastest tier that has it. RAM and local NVMe caches sit in front of the bucket and fill on demand; losing them costs speed, never data.

Read the read path and the hot tier.

Where metadata livesCopy Anchor Link

Data goes to the bucket, but small, hot, transactional state (which manifest is current, which node owns what) needs a consistent store. A single node uses an embedded Raft metastore; larger deployments can opt into TiKV, which also backs Loams Live, Loams SQL and other transactional products. See metadata and TiKV and cluster mode.

One platform, several productsCopy Anchor Link

The same storage, log and metadata layers carry every Loams product. Each badge says how far along it is in the current release: Available ships today, Experimental is built but behind a feature flag or may change more, Planned is designed but not built.

How transactional and analytical work share the same data is its own topic: OLTP, OLAP and HTAP.

Reading pathsCopy Anchor Link

Pick the path that matches what you are building. Each one is a few pages, in order.

App developersCopy Anchor Link

You want search, live data or SQL behind an application, with the least to operate.

1. The bucket is the source of truth
2. Hybrid retrieval
3. Consistency tokens
4. Loams Engine and Loams Live
5. SDKs, then the quickstart
6. The developer guide: install, run locally, first collection, then your language's SDK

Data engineersCopy Anchor Link

You care about formats, pipelines and what else can read the data.

1. The log is the spine
2. Open formats
3. Collections and manifests
4. OLTP, OLAP and HTAP
5. Loams House and Loams SQL

Platform and infrastructure teamsCopy Anchor Link

You will run Loams, size it and keep it safe.

1. Architecture overview
2. Stateless compute and the hot tier
3. Metadata and TiKV
4. Cluster mode
5. Namespaces and tenancy and the security model
6. Self-hosting over Tailscale: the 0.1.0 stack on your laptop or VM, reachable only over your tailnet

Agent buildersCopy Anchor Link

You need memory, tools and runs that survive failure.

1. Hybrid retrieval
2. Loams Graph, for agent memory
3. Loams Durable, for runs that resume
4. Loams Live, for state that pushes updates
5. Loams Studio and Desktop and CLI

Why it is built this wayCopy Anchor Link

Every part of this design is a decision with alternatives we turned down. The decisions page lists them and what each one rules. The full engineering design documents are in the engine repository.