Loams Live

Loams Live is a reactive document database. You write server functions in TypeScript: queries that read, mutations that write. A mutation runs as one transaction and retries itself on conflict. A query a client subscribes to stays live: when a committed write touches anything the query read, Live reruns it and pushes the new result down an open stream. Clients never poll, and every result in one session is taken at the same timestamp, so a screen never mixes two moments. The model is Convex's, rebuilt on TiKV for clusters and on an embedded MVCC store for a laptop.

It is for application state that people watch change: chat, collaborative lists, dashboards, agent run logs, the control data of a console. Use it when you want transactions and push updates without wiring a cache, a queue and a websocket server yourself. Do not use it as your analytics store (it has no SQL, and scans are capped per call), for large blobs (documents are limited to 1 MiB), or for anything that must survive a lost cluster today: per-app backup and restore are designed but not built. And if you already run Postgres and only need change events, Supabase-style logical replication is a shorter path (see the comparison below).

StatusCopy Anchor Link

Loams Live is Available in Loams 0.1.0, the current release: it ships in the release images, is tested in CI, and the 0.1.0 feature table lists no Live-specific known limitation. Two caveats still apply. Its wire package loams.live.v1 is marked unstable, so messages and flags can still change in 0.x. And the design's own production bar (D698: every row of the LV1 exit checklist, including a 24-hour nemesis run, a restore drill and an external security review) has not been met yet.

Stacks on 0.1.0-preview.2 move to 0.1.0 with loams upgrade check --to 0.1.0, then loams upgrade --to 0.1.0, which keeps their data.

ArchitectureCopy Anchor Link

Live runs inside the loams binary. The crates:

The Live role is stateless. Sessions, subscriptions and cached results are derived state that a client's resume rebuilds. The data lives in the store.

How it worksCopy Anchor Link

Documents, tables and keysCopy Anchor Link

An app is one Live database. Its tables hold documents: maps of field to value, where values are null, int64, float64, bool, string, bytes, array or object. int64 and float64 are distinct, so 1 and 1.0 differ. Every document gets _id and _creationTime. An id is 16 random bytes inside the table's key range, written as Crockford base32 with the table id and a checksum, so an id names its table and a forged one is caught. Random ids spread inserts over TiKV regions instead of piling them on one.

Every table has by_id and by_creation_time. A user index lists up to 16 fields, and _creationTime and _id are appended so entries are unique and stably ordered. Index entries are encoded with an order-preserving tuple codec (null, then int64, float64, bool, string, bytes, array), so a range scan in the store returns entries in value order. Documents, index entries, the catalog, the journal and idempotency records all share one key space per app, separated by a one-byte tag.

The defaults in crates/loams-live/src/limits.rs: 1 MiB per document, 1 024 fields, depth 16, 8 192 array elements; per mutation, 8 MiB and 16 000 documents written, 32 000 documents scanned, 4 096 index ranges, 1 s of JavaScript CPU and a 10 s deadline.

Mutations are transactionsCopy Anchor Link

A mutation is one optimistic transaction (D118). The function reads at the transaction's start timestamp, its writes are buffered, and the commit is atomic across regions through TiKV's Percolator two-phase commit. A write conflict aborts the attempt and the runner reruns the whole function at a new timestamp, with jittered backoff, up to 16 attempts as built (the design said 8) or the deadline. Rerunning is safe because functions are deterministic.

The isolation is snapshot isolation, which is weaker than Convex's serializability. Live closes the common case: a document read by id (db.get) is added to the transaction's lock set, so a concurrent write to it is a write-write conflict and one side retries. Index-range reads are not protected, so two mutations that each read a range the other writes into can both commit (write skew). Serializable range reads are open question Q31.

A Mutate call may carry an idempotency key. The record is written in the same transaction, and a retried call returns the recorded commit timestamp and result instead of running again, so a lost acknowledgement never applies a mutation twice. Timestamps are TSO values (physical milliseconds shifted left 18 bits, plus a logical counter) on both backends; they are Live's version numbers.

Commits use plain two-phase commit. Async commit with one-phase commit was the design's default and measured faster in the TiKV spike, but Live stays on two_pc until its checkers pass with the faster mode (Q634).

The commit journalCopy Anchor Link

Invalidation comes from a sharded, sequenced commit journal written inside each mutation's own transaction (D119). The entry lists every document written with its kind and its removed and added index keys. The mutation picks one of the app's shards (64 as built), reads the shard head h, and writes the entry at h + 1 and the head at h + 1 in the same transaction. Two mutations on one shard conflict on the head and one retries, so each shard's sequence is dense and ordered by commit.

That gives a simple correctness argument. The entries visible at tick T and not at the previous tick t are exactly those with head(t) < seq ≤ head(T). If none of them touches a query's read set, nothing the query read changed, and its old result is still the result at T. No timing assumption is needed. The journal costs two extra keys per mutation.

TiKV's own change feed was tested as the alternative. It works on a transactional keyspace from Rust, but needs private client stubs, a hand-written region-by-region subscriber, and about a second of lag before commit order is known. So CDC is a validated fallback, not the invalidation source. A janitor deletes journal entries once every consumer has passed them.

Subscriptions, ticks and TransitionsCopy Anchor Link

Each node runs one subscription manager per app. It ticks on a TSO timestamp T, right after a local commit or else polling with backoff from 20 ms to 200 ms. By default it reads 50 ms behind a fresh timestamp, so commits still in flight do not make it wait on their locks. A tick:

1. reads every shard head and the new entries at T;
2. looks up each written document key and each removed and added index key in the read-set index: interval trees per index plus a set of point keys, mapping to subscription ids;
3. reruns the matched subscriptions at T, at most 16 at once per app. Identical subscriptions (same deployment, function, arguments and identity) share one cached entry, so many clients watching one query cost one rerun;
4. sends one Transition to T to each session whose results changed. Sessions with no change get nothing.

A query's read set is ranges, not document lists, so an insert into a scanned range is caught. A range that stopped at its limit is recorded up to the last key returned, so inserts past a full page do not invalidate it. If writes arrive faster than reruns, the next tick reruns at a newer timestamp: a client can see fewer versions, never an inconsistent one. A subscription reruns at most once per 50 ms. As a safety net, every subscription is also rerun every 5 minutes and compared; a difference is a missed invalidation, which is logged, counted and repaired.

The sync API (D121) is a server stream plus unary calls, never bidirectional, because browsers cannot stream full duplex over fetch. Watch opens a session and streams Transitions, with a heartbeat every 15 s. ModifyQuerySet adds and removes queries, Query is a one-shot read, Mutate returns the commit timestamp, and Authenticate replaces a session's identity. A session's version is the triple (query-set version, identity version, ts). A client applies a Transition only if its start equals the client's current version; on a gap it resumes and gets full results. Mutate returns commit_ts, so a client keeps an optimistic update until its session reaches that timestamp.

Server functionsCopy Anchor Link

Functions are ES modules that import from loams:server. This one is lightly trimmed from crates/loams-live-js/tests/functions.rs:

import { query, mutation, internalQuery } from "loams:server";

export const messages = {
  send: mutation({
    handler: async (ctx, args) => {
      return await ctx.db.insert("messages", { channel: args.channel, body: args.body });
    },
  }),
  list: query(async (ctx, args) => {
    return await ctx.db.query("messages").order("desc").take(10);
  }),
  count: internalQuery(async (ctx) => (await ctx.db.query("messages").collect()).length),
};

These deploy as messages:send, messages:list and messages:count. query and mutation are public. internalQuery and internalMutation are reachable only by platform principals. Reads use ctx.db.get, ctx.db.query(...).withIndex(...), order, take, first, collect and paginate. Pagination cursors carry an HMAC under a per-app key, so a client cannot forge a cursor into another range. Arguments can have v.* validators, checked in Rust before any JavaScript runs. ctx.auth.getUserIdentity() returns the signed-in end user or null. Six built-ins, _system:get, query, tables, insert, patch, replace and delete, work before any deploy; the console's data views use them.

Functions run in QuickJS through rquickjs (D120). Each call gets a fresh context, so module state cannot leak between calls. Queries and mutations must be deterministic: Date.now() is the transaction's start time, Math.random() is a seeded PRNG, crypto.getRandomValues() and crypto.randomUUID() throw, local time is UTC, and there are no timers, network or filesystem. One known limit: transcendental Math functions use the host's C library, so nodes built for different platforms could disagree in the last bit.

There are two isolation modes (D681). in_process, the default, is for loams dev, Loams Desktop and single-tenant self-hosting, where the code is the operator's own. isolated runs functions in loams live-worker subprocesses that speak a framed protobuf protocol over stdio. Each worker runs under Landlock with no filesystem or network rights, no new privileges, three file descriptors and a memory limit. A crash fails the call with live_worker_crashed and the worker respawns. --live-tenancy multi refuses to start without --live-isolation isolated, which is Linux only.

Deploys, validators and online indexesCopy Anchor Link

A deployment is an immutable record: a bundle hash, a schema, who made it, a message and a parent (D683). Activation is a compare-and-swap of the app's deployment pointer. Deploy carries the deployment the client built against and fails with live_deploy_conflict if another deploy got there first. Rollback is ActivateDeployment of an earlier record, so it goes through the same path and re-applies the earlier schema. At the activation tick every subscription reruns once, so a session never sees results from two deployments in one Transition. The activation is written to the journal as a catalog entry, so every node switches at the same tick.

Validators are enforced on every write, _system:* included (D684). A deploy that tightens a validator first enforces it for new writes, then scans existing documents and fails on a violation, unless the table sets schemaValidation: false. Loosening applies at once.

Index changes on live tables are two-phase (D685). The index first becomes write_only, so new mutations maintain it while queries refuse it (live_index_not_ready). Every mutation checks a fresh timestamp before prewrite and refuses to commit past its 10 s lifetime. After twice that lifetime, no mutation that read the old table definition can still commit. A backfill then walks the table in batches that lock the document keys they index, checkpointing as it goes, and the index becomes ready. Drops reverse the order.

Apps, keyspaces and nodesCopy Anchor Link

On TiKV every app gets a dedicated API v2 keyspace, named by its id (loams_live_a<id>); the directory of apps lives in the system keyspace loams_live_system (D690). A request names its app with the loams-live-app header. A node starts an app's runner, subscription manager, tailer and runtime on the first request and stops them after 10 minutes idle. A deleted app refuses calls and its data is purged after 7 days. The original design (D122) also has shared keyspaces for small apps under a key prefix. They are not built, because each keyspace costs at least one region and the cluster's limit is still to be measured (Q36).

Several nodes can serve one store (D693, LV1 Task 24). Each node registers with --live-node and --live-advertise. A ModifyQuerySet or Authenticate that reaches the wrong node is forwarded to the session's owner with the caller's own credential, so a load balancer needs no affinity. If the owner is gone, the call fails with session_gone and the client resumes. On SIGTERM a node drains: readiness goes false, streams close with node_draining, and running mutations finish. As built, a Live "cluster" is several standalone engines on one store. Live refuses the engine's --cluster mode.

Loams runs its own MVCC garbage collection for TiKV keyspaces, because TiKV v8.5.8 reads only a cluster-wide safe point and nothing else advances it without TiDB. That is a detail of metadata on TiKV.

The embedded storeCopy Anchor Link

loams dev and Loams Desktop run Live on an embedded store (D691, D692), because a TiKV playground needs gigabytes of memory and does not run natively on Windows. It is one redb file at <data-dir>/live/store.redb with versions keyed key ‖ !commit_ts. It has a timestamp oracle that survives restarts, snapshot isolation with first-committer-wins, group commit and GC. The same conformance suite runs against both backends, and so do the reactive checker and the transaction checker. Production means TiKV: the embedded store is not supported there (Q626).

AuthCopy Anchor Link

Live has two kinds of principal (D686). Platform principals (Loams tokens and API keys) deploy, administer and call internal and _system:* functions. They are gated by the scopes live:call, live:read, live:write (or per table), live:deploy and live:admin. A credential bound to an environment reaches only that environment's app (live_app_mismatch). End users bring tokens from OIDC issuers configured per app and may call only public functions. Live verifies credentials and never issues them. Authenticate swaps a session's identity: the same user only extends it, while a different user bumps the identity version and reruns every query in one Transition. Subscriptions are keyed by identity, so a result is never shared across users (D687). Sessions close 60 s after their token expires, on revocation, and after 24 h for rotation.

Key decisionsCopy Anchor Link

All decisions are in the decision index. Several that the code implements are still marked Proposed in the log.

For multi-tenant hostile code, the owner's 2026-10-15 sandbox direction (D993, answering Q628) says the Landlock-and-seccomp worker is not enough on its own. It is to be stacked inside gVisor or a microVM, and D1003 moves Live onto the shared loams-sandbox crate.

What is compatibleCopy Anchor Link

• Protocols. Connect (JSON and binary), gRPC and gRPC-Web over HTTP/1.1 and HTTP/2, on the engine's main port, from connectrpc 0.9.1 on the server.
• TypeScript. @loams/live carries the generated messages and the LiveService and LiveAdminService descriptors. It is built with @bufbuild/protobuf 2.16.0 for @connectrpc/connect 2.2.0. @loams/client (Node 22 or later, browsers, Deno, Bun) exposes them as loams.live (watch, modify, authenticate), loams.tables (query, mutate) and loams.live.admin. The 0.1.0 release lists both for npm at version 0.1.0 (a registry publishes once it is enabled for the project). There is no hand-written reactive client yet: you consume the Watch stream yourself.
• Other SDKs. The Swift SDK wraps LiveAdminService as client.live.admin. The Python, Go, Rust, C# and C++ SDKs get the generated stubs through their facades. Reactive layers for them are R2 and R3 work.
• Stores. TiKV and PD v8.5.8 with storage.api-version = 2, the version CI runs in a tiup playground. Live talks to them through Loams' fork of tikv-client (ostrium-labs/client-rust, pinned by revision). The embedded backend uses redb 4.
• Convex code. Not compatible on purpose. Live borrows concepts, not the wire protocol, the function API names or any code (§20 non-goals), though the shapes (query, mutation, ctx.db, v.*) will look familiar.

Compared with the alternativesCopy Anchor Link

Convex is the model Live follows, and it is far ahead. It is a mature product with a hosted cloud, a dashboard, React and other clients, actions, scheduling, file storage and search, and an open-source backend you can self-host (its licence is FSL-1.1-Apache-2.0, which becomes Apache-2.0 only after a delay; that is why Loams reads it for concepts only). It promises serializability; Live does not yet for range reads. Choose Convex today if you want the developer experience without operating anything. Live's case is narrower: the same model inside a stack you already run, with Apache-2.0 code, TiKV underneath, and (planned, not built) Live tables that feed Loams' hybrid retrieval without an ETL job.

Supabase Realtime solves a different problem. It streams row changes from Postgres over websockets, and your client merges them into whatever it shows. It does not rerun your query, so joins, filters and limits are your client's job, but you keep full Postgres and SQL. If your data is already in Postgres, that is usually the better fit. Live gives you consistent query results at one timestamp in exchange for its own data model and no SQL.

Firestore listeners are closest in feel: a listener on a query gets new snapshots. Firestore adds offline persistence and Google's operations. It is a managed Google service only.

What is missing todayCopy Anchor Link

From the LV1 plan and design §45:

• Backup and restore (Tasks 29 to 31). Per-app snapshot plus continuous journal export with point-in-time restore is designed (D694) but not built, and BR's coverage of transactional keyspaces is unverified (Q37, Q627). Keep your own copy of anything you cannot recreate.
• Operations (Tasks 25 to 28, 32, 33). There is no 24-hour nemesis run, no Prometheus metrics or traces beyond logs, no quotas, no reference deployment and no measured performance gates. The targets in §45 §15 are targets, not results.
• Clients (Tasks 34 to 37). There is no @loams/live reactive client with resume and optimistic updates, no @loams/live-react hooks, and no @loams/live-cli for bundling and deploying. You bundle with your own tool and call LiveAdminService.Deploy.
• Release gate (Tasks 38 to 40). Docs, the threat model and fuzzing, an external review (Q635), and dropping unstable from loams.live.v1.
• Function API. No actions (fetch) and no scheduled functions (Q625, R2). No serializable range reads (Q31). No full-text or vector search from functions until the collections bridge (D129, R3).
• Tenancy. No shared keyspaces for small apps (R2). For hostile multi-tenant code, the worker sandbox must sit inside gVisor or a microVM first (D993).
• Commit mode. two_pc until the checkers pass with async commit (Q634).

Try itCopy Anchor Link

Build the engine and run it on one machine (from the repository README). Live starts on the embedded store with no PD:

git clone https://github.com/ostrium-labs/loams.git
cd loams
cargo run --release -p loams -- dev

Live is served on the engine's main port (--listen, port 8080 by default, loopback only) beside the other loams.*.v1 services. It speaks Connect JSON, so curl works. This insert and read are the calls the live_dev test makes, with LOAMS_URL set to the engine's address:

curl -s "$LOAMS_URL/loams.live.v1.LiveService/Mutate" \
  -H 'content-type: application/json' \
  -d '{"function":"_system:insert","args":{"objectValue":{"fields":{
        "table":{"stringValue":"notes"},
        "fields":{"objectValue":{"fields":{"n":{"int64Value":"7"}}}}}}}}'

curl -s "$LOAMS_URL/loams.live.v1.LiveService/Query" \
  -H 'content-type: application/json' \
  -d '{"function":"_system:tables","args":{"objectValue":{}}}'

To run Live on TiKV instead, use a build with the live-tikv feature and pass --live-store tikv://<pd>[,<pd>]/<keyspace>. The self-hosted compose stack in deploy/release does this: its engine image is built with live,live-tikv and runs PD and TiKV beside it. Follow Running Loams 0.1.0 to start it on your tailnet, then open the console's Live pages, or call Live from @loams/client:

npm i @loams/client @loams/live

Useful flags on loams dev and loams standalone: --live-app (the default app, dev), --live-isolation in_process|isolated, --live-tenancy single|multi, and --live-node and --live-advertise for several nodes. --no-live turns Live off.

SourceCopy Anchor Link

• Crates: loams-live, loams-live-js, loams-live-proto, loams-kv
• Protos: proto/loams/live/v1
• TypeScript: sdks/typescript/packages/live
• Correctness: tests/reactive_checker.rs checks every pushed result against a fresh read at its timestamp; tests/txn_checker.rs checks Elle-style histories for snapshot isolation
• Design: §20 Loams Live: a reactive database on TiKV, §45 Loams Live: production readiness, §56 the sandbox
• Plans: R1 reactive core, LV1 production readiness
• Related Learn pages: metadata and TiKV, OLTP, OLAP and HTAP, stateless compute, namespaces and tenancy