What we're building, and how.
What Loams is, how each part works, and the open-source projects it stands on. Every post says what is available today and what is still being built.
Series
Opinions
The decisions behind Loams and why we made them: a KV as the transactional spine, the bucket as the database, the log as the spine, HTAP, protocols, forks, and what Loams is bad at today.
- A distributed KV is all you need10 min
- Your bucket is the database9 min
- The log is the spine9 min
- OLTP, OLAP, HTAP, LTAP: the acronyms and what actually matters11 min
- Five clusters, one bucket9 min
- Drop-in, not rip-and-replace10 min
- Forking Neon, TiDB and Grafeo: owning your dependencies11 min
- What Loams is bad at today11 min
The Loams platform
One post per pillar: the vision, the engine, the reactive database, durable execution, functions, jobs, SQL and self-hosting.
- Loams: the AI-native cloud in your bucket9 min
- The engine: hybrid retrieval, streams, Iceberg and the wire APIs9 min
- Loams Live: a reactive database on TiKV10 min
- Durable execution: Resonate, embedded9 min
- Functions billed on CPU time7 min
- Jobs on Loams: Celery, BullMQ, PySpark and Flink SQL7 min
- Postgres and MySQL on your bucket4 min
- Self-hosting Loams with GitOps7 min
What Loams is built on
Deep dives on the open-source projects under Loams: how each works, why we chose it, how we use it, its license and its limits.
- Resonate: durable promises, and the server we embed7 min
- TiKV and PD: Raft, Percolator, keyspaces and our client fork8 min
- RustFS: the object store under a self-hosted Loams4 min
- Apache DataFusion and Arrow: the query engine under every Loams read5 min
- Lance: the columnar format that holds Loams’s documents and vectors5 min
- Tantivy and Quickwit’s splits: full-text search on object storage5 min
- Apache Iceberg, iceberg-rust and Lakekeeper: tables any engine can read5 min
- Dapr: the API surface, and secrets from every cloud6 min
- workerd, wasmtime and gVisor: three ways to run code you did not write6 min
- OpenFGA: relationship-based authorization for Loams5 min
- Beside Loams, not inside it: Sail, RisingWave, Neon and MySQL7 min
- The small crates that carry a lot: SlateDB, foyer, openraft, redb, qdrant-edge and connect-rust6 min
The protocols Loams speaks
One post per wire protocol and API: how it frames, handshakes and authenticates, why Loams speaks it, what we implement and what we do not, and the gotchas.
- The Postgres wire protocol: startup, TLS, SCRAM and SNI routing10 min
- The MySQL wire protocol: the greeting, capability flags and caching_sha2_password8 min
- Arrow Flight SQL and ADBC: SQL that answers in Arrow9 min
- The Qdrant API: REST, gRPC and the universal query10 min
- The Elasticsearch REST API and a Query DSL subset9 min
- Connect RPC: one handler for gRPC, gRPC-Web and plain HTTP9 min
- The S3 API: the bucket contract Loams depends on9 min
- The Git smart protocol and remote helpers: how stock git reaches a bucket10 min
- MCP: how agents call Loams, and how Loams agents call each other8 min
- The ClickHouse HTTP interface: how Loams House answers curl9 min
- The Kafka wire protocol: what it takes to be a broker8 min
- OTLP: OpenTelemetry logs into Loams streams6 min
- The Iceberg REST catalog: one pointer per table, over HTTP7 min
- The Loams Data API: SQL over HTTP, designed for retries and agents8 min
- CloudEvents: one envelope for every event in Loams7 min
Research
Surveys of a field with primary sources: what the projects and papers say, where they agree, what they get wrong, and where Loams stands, with every Loams claim labelled by its release status.
- Where the WAL lives: databases on object storage in 202620 min
- Hybrid retrieval fusion: RRF, convex combination, and rerankers27 min
- What agents need from infrastructure: memory as data, durable runs, sandboxes with policy, and a harness that does not trust the model17 min
- How Postgres runs on object storage: Neon, Aurora and our hard fork19 min
- Why the Loams engine never faces the public internet19 min
- A software factory you own14 min
- Memory, harnesses and fewer tokens: what grew in open-source agent and data infrastructure, autumn 202615 min
- The harness is an object: places, capabilities and a jailer that doesn't trust the model16 min
- Git on the bucket: what we learned from Piper, CitC, Enroute and Scion15 min
- Agent memory as data: what HydraDB, Mem0, Graphiti and Zep taught us13 min
Teardowns
How other systems work, read from their public code and documentation, and what Loams takes from them.
Where the WAL lives: databases on object storage in 2026
A sourced survey of who builds databases on S3 and how. turbopuffer, LanceDB, Neon, WarpStream, S2, SlateDB, Quickwit, Iceberg, Snowflake, Databricks, TiDB X, Bufstream and AutoMQ, compared on write path, caching, consistency, cost, tenancy and formats. Then where Loams fits, and where it does not.
Hybrid retrieval fusion: RRF, convex combination, and rerankers
What the papers and the shipping engines say about combining BM25 and vector results, why score normalization is where most hybrid setups go wrong, what k in RRF actually does, how to evaluate a fusion choice, the production details (per-signal top-k, filtering, pagination), and exactly what Loams does and does not do today.
What agents need from infrastructure: memory as data, durable runs, sandboxes with policy, and a harness that does not trust the modelWhat agents need from infrastructure: memory as data, durable runs, sandboxes with policy, and a harness that does not trust the model
A survey with primary sources of agent memory (MemGPT and Letta, Mem0, Zep and Graphiti, supermemory, LangGraph), durable execution (Temporal, Restate, Inngest, DBOS, Resonate), sandboxes (Firecracker, gVisor, E2B, Modal, Daytona, Cloudflare), coding-agent harnesses and the security research on agents. Then what most stacks get wrong, and where Loams stands, with each piece labelled by its status.
How Postgres runs on object storage: Neon, Aurora and our hard forkHow Postgres runs on object storage: Neon, Aurora and our hard fork
How Neon splits Postgres into a stateless compute, a Paxos WAL service and a pageserver that keeps layer files in a bucket; how Aurora, AlloyDB, OrioleDB, Xata, Nile, Lakebase and PlanetScale answer the same question; and what Loams changed in Loams Postgres, its hard fork of Neon, and why.
Why the Loams engine never faces the public internetWhy the Loams engine never faces the public internet
Self-hosting a data platform on your own tailnet. The threat model, what zero trust and WireGuard actually give you, a decade of exposed Elasticsearch, MongoDB, Redis and ClickHouse instances, the four auth modes, the hosted console that talks to your engine over your tailnet, BYOC over SSH, and what is still not hardened in 0.1.0.
A distributed KV is all you needA distributed KV is all you need
Why one TiKV cluster, with Raft, Percolator transactions and keyspaces, is the transactional spine under Loams' metadata, Live, durable state and Loams SQL. And the four things a KV is bad at, where we use the bucket or a log instead.
The Postgres wire protocol: startup, TLS, SCRAM and SNI routingThe Postgres wire protocol: startup, TLS, SCRAM and SNI routing
How a Postgres connection is framed, encrypted and authenticated, why SSLRequest breaks ordinary SNI routers, how Loams serves the protocol twice (real Postgres on Loams Postgres, and a read-only listener over collections), and how the planned loams-gateway routes by SNI without taking PgDog's keys.
The MySQL wire protocol: the greeting, capability flags and caching_sha2_passwordThe MySQL wire protocol: the greeting, capability flags and caching_sha2_password
How a MySQL connection is framed and negotiated, why the server speaking first makes routing hard, how loams-sqlgate greets, authenticates and relays for Loams SQL on TiDB, and the small read-only MySQL listener over collections.
Arrow Flight SQL and ADBC: SQL that answers in ArrowArrow Flight SQL and ADBC: SQL that answers in Arrow
How Arrow Flight frames data over gRPC, how Flight SQL packs SQL into it, where ADBC sits on top, and what the Loams engine serves today: read-only SQL over collections, bulk ingest through DoPut, and self-contained tickets any node can run.
The Qdrant API: REST, gRPC and the universal queryThe Qdrant API: REST, gRPC and the universal query
How Qdrant's REST and gRPC APIs are shaped, why Loams serves them unchanged so LangChain and LlamaIndex vector stores work without a code change, how loams-qdrant maps points onto Loams collections, and every place we deliberately differ.
The Elasticsearch REST API and a Query DSL subsetThe Elasticsearch REST API and a Query DSL subset
How Elasticsearch's REST API negotiates versions, frames bulk writes as NDJSON and expresses search in the Query DSL, why Loams serves a subset chosen by what AI frameworks actually send, how loams-es compiles it, and where it parts from Elasticsearch 8.19.
Connect RPC: one handler for gRPC, gRPC-Web and plain HTTPConnect RPC: one handler for gRPC, gRPC-Web and plain HTTP
How the Connect protocol frames unary and streaming calls, how it differs from gRPC and gRPC-Web, and why every Loams application API is a Connect service on one port that curl, grpcurl and a browser can all call.
The S3 API: the bucket contract Loams depends onThe S3 API: the bucket contract Loams depends on
Loams does not serve S3, it consumes it, and every durable byte depends on a handful of S3 behaviours being exactly right. SigV4, ranged reads, multipart, conditional writes and strong consistency, what Loams requires of any S3 store, and how we test it.
The Git smart protocol and remote helpers: how stock git reaches a bucketThe Git smart protocol and remote helpers: how stock git reaches a bucket
pkt-lines, capability advertisement, protocol v2's ls-refs and fetch, receive-pack for pushes, and the remote-helper protocol that lets `git push` write straight to object storage through git-remote-loams. What exists today, and what Smart HTTP and monorepo mode add.
MCP: how agents call Loams, and how Loams agents call each otherMCP: how agents call Loams, and how Loams agents call each other
The Model Context Protocol's JSON-RPC messages, handshake, transports and auth; the stdio MCP server Loams' agent daemon injects into every coding-agent run today; and the stateless HTTP surface planned for the engine.
The ClickHouse HTTP interface: how Loams House answers curlThe ClickHouse HTTP interface: how Loams House answers curl
ClickHouse's HTTP interface is the simplest database protocol in wide use, a query in a POST body and the answer in the format you ask for. How it works, why Loams House speaks it before ClickHouse's native protocol, and what the House accepts, refuses and does differently.
The Kafka wire protocol: what it takes to be a brokerThe Kafka wire protocol: what it takes to be a broker
How Kafka frames requests, negotiates versions with ApiVersions, authenticates with SASL and lays out record batch v2. Why Loams stores Kafka's batch format already, why the Loams-side gateway is deferred in favour of contributing to Apache Iggy's, and what a Kafka front end over Loams streams has to get right. Planned.
OTLP: OpenTelemetry logs into Loams streamsOTLP: OpenTelemetry logs into Loams streams
How the OpenTelemetry Protocol carries logs, traces and metrics over gRPC and HTTP, what partial success and its retry rules mean for a receiver, and Loams' plan to take OTLP logs into streams so Fluent Bit, the Collector and Vector ship to Loams with no plugin. Planned.
The Iceberg REST catalog: one pointer per table, over HTTPThe Iceberg REST catalog: one pointer per table, over HTTP
The Iceberg REST catalog protocol is how every engine finds, reads and commits an Iceberg table. How /v1/config, loadTable, credential vending and commits with requirements and updates work, how Loams House already speaks it as a client to Lakekeeper, and the plan to make Loams tables readable by DuckDB, Spark, Trino and ClickHouse through it.
The Loams Data API: SQL over HTTP, designed for retries and agentsThe Loams Data API: SQL over HTTP, designed for retries and agents
An open, Apache-2.0 spec for running SQL against Postgres, Loams SQL and the engine over Connect, with no driver and no held connection. Statement batches, batons for interactive transactions, idempotency keys recorded inside the transaction, typed retry-aware errors, dry runs, budgets, Arrow results, live queries and a safety profile for agents. Draft v0, not implemented yet.
CloudEvents: one envelope for every event in LoamsCloudEvents: one envelope for every event in Loams
CloudEvents 1.0 defines what an event is (id, source, type, specversion) and how it rides on HTTP and Kafka in binary and structured modes. How Loams streams ingest and serve CloudEvents today, deduplicating by source and id, why we wrote our own codec, and the Loams profile that the rest of the platform is converging on.
A software factory you ownA software factory you own
Google and Meta built their own software factories. Everyone else rents one in pieces, and coding agents are about to rent the loop as well. The open-source parts are good enough to own now. What we ship in Loams today, what we are designing, and what we are not sure about.
Memory, harnesses and fewer tokens: what grew in open-source agent and data infrastructure, autumn 2026Memory, harnesses and fewer tokens: what grew in open-source agent and data infrastructure, autumn 2026
A field survey of the open-source repositories that gained the most attention from August to October 2026, by category. Agent harnesses, memory layers, code graphs, MCP, graph engines on object storage, Postgres for agents, local tools. Momentum with sources and dates, why each is growing, what it means if you build on it, and where Loams fits.
The harness is an object: places, capabilities and a jailer that doesn't trust the modelThe harness is an object: places, capabilities and a jailer that doesn't trust the model
What Loams took from Endo and Jason Duncan's essay "The Harness Is an Object", how forty years of object-capability security turn its algebra into rules a machine can check, and the proposed Loams Jailer, a deterministic policy layer with an LLM overseer that is reasoning-blind and may only tighten. Both designs are Proposed; nothing here ships yet.
Git on the bucket: what we learned from Piper, CitC, Enroute and ScionGit on the bucket: what we learned from Piper, CitC, Enroute and Scion
Google's Piper and CitC, Cursor's Continuity, Enroute's Raft ledger and layers, and Anthropic's Scion, read side by side for one question each. Where a push is ordered, what a workspace is, who merges, and what sits between a laptop and the store. Then the five changes Loams Git took from them (D1025 to D1029), and what ships in 0.1.0 and what does not, each marked with its status.
Agent memory as data: what HydraDB, Mem0, Graphiti and Zep taught usAgent memory as data: what HydraDB, Mem0, Graphiti and Zep taught us
What we took from four systems while designing Loams Memory. Mem0 on why judging memories at write time is costly, Graphiti and Zep on episodes and facts with two clocks, and HydraDB on keeping a graph index fresh from its own log. Then how those ideas land on collections, a linked graph and the Loams log, what we deliberately left out, and where each piece stands.
Teardown: how reactive and sync databases work under the hoodTeardown: how reactive and sync databases work under the hood
How Convex, Supabase Realtime, Firestore, Zero and Replicache, ElectricSQL, PowerSync, InstantDB, Triplit and LiveStore turn a query into a subscription, work out what a write invalidates, and what they promise when the network fails. Reconstructed from their public code, docs and papers, then compared with Loams Live, including where Live is behind.
Coding-agent harnesses, taken apart: Codex, Claude Code, Cursor, OpenCode, Aider, Goose, Cline and T3 CodeCoding-agent harnesses, taken apart: Codex, Claude Code, Cursor, OpenCode, Aider, Goose, Cline and T3 Code
How coding-agent harnesses actually work, read from their public code and documentation. The loop, tools and approvals, per-OS sandboxes, compaction, edit formats, checkpoints and worktrees, MCP, subagents, telemetry and failure modes, and then where Loams Studio and loams-agentd fit.
Teardown: nine engines built on a bucket, and the one trick each gets rightTeardown: nine engines built on a bucket, and the one trick each gets right
How turbopuffer, WarpStream, S2, SlateDB, Neon, Lance, Quickwit, AutoMQ and Bufstream turn object storage into a database, a log or a search index. The architecture, the key trick and the published numbers for each, what conditional writes on S3, R2, Tigris and self-hosted stores really give you, and the ideas we are considering for Loams.
Your bucket is the databaseYour bucket is the database
Why Loams treats object storage as the only source of truth and every server as a cache. The cost and durability arithmetic with its sources, what you give up in latency, how hot tiers and WAL classes buy it back, and where we bend the rule.
The log is the spineThe log is the spine
In Loams every write is a log entry and every collection, table and graph is a materialization of that log, maintained by links that commit their applied offset with the data. Why that beats change data capture bolted on afterwards, what a consistency token buys, and what the design costs.
OLTP, OLAP, HTAP, LTAP: the acronyms and what actually mattersOLTP, OLAP, HTAP, LTAP: the acronyms and what actually matters
Precise definitions of four overloaded acronyms, the four shapes HTAP takes in shipping products, what Snowflake, Databricks, TiDB, SingleStore, Neon, ClickHouse and DuckDB each get right and miss, and where Loams stands today, including what it does not have yet.
Five clusters, one bucketFive clusters, one bucket
A typical AI application runs Elasticsearch, Qdrant, a graph database, Kafka and a workflow engine, with the same data copied into each. Why we think hybrid retrieval belongs in one planned query over one copy, and where the specialized systems still win.
Drop-in, not rip-and-replaceDrop-in, not rip-and-replace
Why Loams speaks the Qdrant, Elasticsearch, Postgres, MySQL and Arrow Flight SQL protocols instead of asking you to adopt a new API first, how we keep each one scoped by test suites, and what compatibility costs us.
Forking Neon, TiDB and Grafeo: owning your dependenciesForking Neon, TiDB and Grafeo: owning your dependencies
Why Loams runs PD and TiKV unmodified, carries one cherry-pick on TiDB, keeps a patch queue on Grafeo and hard-forked Neon as Loams Postgres. What each fork costs, the rules that keep them honest, and why we did not fork ClickHouse.
What Loams is bad at todayWhat Loams is bad at today
A candid list for Loams 0.1.0. It is a 0.x release with one way to run it, a security model that is solid at the edge and trusting inside, a single region, half of the original pitch still planned, a Postgres WAL service that misses its targets, and unsigned desktop builds. What each one means for you, and what we are doing about it.
Loams: the AI-native cloud in your bucketLoams: the AI-native cloud in your bucket
An AI application today runs five or six stateful systems. Loams puts retrieval, application data, durable workflows and jobs on one platform, with object storage as the ground it all stands on.
The engine: hybrid retrieval, streams, Iceberg and the wire APIsThe engine: hybrid retrieval, streams, Iceberg and the wire APIs
How Loams combines vector, sparse and full-text retrieval on a bucket, with native and compatibility APIs, streams and the next graph/analytics integration steps.
Loams Live: a reactive database on TiKVLoams Live: a reactive database on TiKV
Queries that stay live and push new results, mutations that are transactions, server functions, a protobuf sync API, and a change feed that makes application data searchable. How it works, and what is built.
Durable execution: Resonate, embeddedDurable execution: Resonate, embedded
Loams links the Resonate server into its own binary, so the official Resonate SDKs get durable promises, retries, schedules and human approvals from the system that holds the agent's memory. Here is how, and the patterns we build on it.
Functions billed on CPU timeFunctions billed on CPU time
An agent that computes for 10 ms and waits 20 s on a model should pay for 10 ms. Our proposal for Loams Functions, with workerd, wasmtime and gVisor tiers and Resonate for long waits.
Jobs on Loams: Celery, BullMQ, PySpark and Flink SQLJobs on Loams: Celery, BullMQ, PySpark and Flink SQL
Our proposal for running the job frameworks teams already use on Loams with a configuration change, on one Rust queue core over TiKV, with durable steps from Resonate when you want them.
Postgres and MySQL on your bucketPostgres and MySQL on your bucket
Read-only SQL access, Neon-compatible WAL and the integration work behind Loams Postgres and Loams SQL.
Self-hosting Loams with GitOpsSelf-hosting Loams with GitOps
How the whole stack is meant to deploy from one Git repository, from a laptop cluster to your own data centre, on RustFS, the TiKV operator, a Loams operator and Argo CD. And how the line between open source and the managed cloud went away.
Resonate: durable promises, and the server we embedResonate: durable promises, and the server we embed
What Resonate is, how durable promises and replay work, why Loams chose it over Temporal and the alternatives, how we link its Rust server into our binary through a fork, and its license and limits.
TiKV and PD: Raft, Percolator, keyspaces and our client forkTiKV and PD: Raft, Percolator, keyspaces and our client fork
How TiKV replicates and transacts, what PD does, why API v2 keyspaces made TiKV Loams’ transactional store, what its change feed can and cannot do for us, and why we carry a fork of the Rust client.
RustFS: the object store under a self-hosted LoamsRustFS: the object store under a self-hosted Loams
Loams’ source of truth is a bucket, so the bucket has to keep its promises. What RustFS is, why it replaced MinIO as Loams’ default self-hosted store, what Loams needs from any S3 store, and what we still have to verify.
Apache DataFusion and Arrow: the query engine under every Loams readApache DataFusion and Arrow: the query engine under every Loams read
Every Loams read, whether a Qdrant query, an Elasticsearch search or SQL, compiles to a DataFusion plan over Arrow batches. How DataFusion works, what Loams adds to it, why we embed it, and the version lockstep that comes with it.
Lance: the columnar format that holds Loams’s documents and vectorsLance: the columnar format that holds Loams’s documents and vectors
How Lance lays out fragments, versions and indexes for fast random access on object storage, why Loams stores collections in it instead of Parquet, the detached versions we commit, and the churn we pin against.
Tantivy and Quickwit’s splits: full-text search on object storageTantivy and Quickwit’s splits: full-text search on object storage
How Tantivy indexes and scores text, how Quickwit's split format makes an index readable from a bucket with one ranged GET, why Loams vendors Quickwit's code instead of running Quickwit, and how we keep BM25 scores independent of the split layout.
Apache Iceberg, iceberg-rust and Lakekeeper: tables any engine can readApache Iceberg, iceberg-rust and Lakekeeper: tables any engine can read
How Iceberg tables commit on object storage, what iceberg-rust can and cannot write today, why Lakekeeper is Loams’ catalog, and what Loams plans to add on top for AI analytics. All of it planned.
Dapr: the API surface, and secrets from every cloudDapr: the API surface, and secrets from every cloud
What Dapr is and how its sidecar and components work, why Loams plans to serve a subset of Dapr's API from a Rust server instead of running a sidecar per function, and how tenant secrets would flow through Dapr's secrets building block.
workerd, wasmtime and gVisor: three ways to run code you did not writeworkerd, wasmtime and gVisor: three ways to run code you did not write
The three isolation technologies behind Loams’ proposed function tiers. How V8 isolates, WebAssembly and a user-space kernel each contain code, what each costs, how each can be metered, and why workerd never runs two tenants in one process.
OpenFGA: relationship-based authorization for LoamsOpenFGA: relationship-based authorization for Loams
How OpenFGA's Zanzibar-style model answers "may this user do this to that", why Loams plans it as its fine-grained authorizer beside built-in roles, the tenant fence we add to Lakekeeper's model, and why tuple writes go through an outbox.
Beside Loams, not inside it: Sail, RisingWave, Neon and MySQLBeside Loams, not inside it: Sail, RisingWave, Neon and MySQL
Sail and RisingWave designs, plus the implemented WAL, routing and deployment foundations behind Loams Postgres and Loams SQL.
The small crates that carry a lot: SlateDB, foyer, openraft, redb, qdrant-edge and connect-rustThe small crates that carry a lot: SlateDB, foyer, openraft, redb, qdrant-edge and connect-rust
Six Rust libraries inside Loams that get less attention than DataFusion or Lance but hold up the primary-key index, the cache, the metastore, the HNSW hot tier and every protobuf API. What each does, how Loams uses it, and its limits.
Crash gates: how we test kill -9 at every stepCrash gates: how we test kill -9 at every step
Named failpoints, a store fault matrix and a seeded linearizability simulation. This is what "no acknowledged write is ever lost" means in Loams’ test suite.
One manifest, two formats: Lance + TantivyOne manifest, two formats: Lance + Tantivy
How Loams keeps a Lance dataset and a set of Tantivy splits mutually consistent on S3, with one compare-and-swap as the only commit point.
Why a retrieval engine should live on object storageWhy a retrieval engine should live on object storage
Most of an AI app's search data is cold most of the time. Here is why Loams keeps every byte in your bucket and treats compute as a cache.