Loams SQL

Loams SQL is the MySQL-compatible database in Loams. The SQL layer is TiDB, run as stateless tidb-server processes. The data lives in the same TiKV cluster that already holds Loams' metadata and Loams Live, and every database branch gets its own TiKV keyspace. The goal is a "Neon-like MySQL": compute that starts and stops on demand, storage that is shared, and branches made by the control plane.

Use it when you have an application that speaks MySQL (an ORM, a framework, an existing schema) and you want it inside the same stack as the rest of Loams, on infrastructure you run. Do not use it today if you depend on stored procedures, triggers, SERIALIZABLE isolation or gap locks: TiDB does not have them, and Loams SQL takes TiDB's semantics as they are. And keep a copy of data you cannot recreate: 0.1.0 runs one TiDB process with a passwordless root, and the parts that make it serverless are still being built.

StatusCopy Anchor Link

Loams SQL is Available in Loams 0.1.0, the first release (2026-10-18). The feature table in docs/release/preview.md lists it as "MySQL-compatible SQL on a patched TiDB (v8.5.8-loams.1) over TiKV". Available is narrower than the design below, so here is exactly what ships and what does not:

The release notes' known limitations say it directly: the Loams SQL gate is not in this release, and the console's SQL page still shows demonstration data. What you get in 0.1.0 is a real, keyspace-mode TiDB on the stack's TiKV that any MySQL client can use from inside the compose network. The two previews before it (2026-10-12 and 2026-10-15) shipped the same tidb service.

Do not confuse Loams SQL with the engine's mysql-wire listener. That listener serves read-only SQL over Loams collections (the analytics side of the engine; its removal is proposed, SQ1 Task 29 and Q663). It never touches TiDB.

ArchitectureCopy Anchor Link

Three layers make up the design (design §47 §1, §3):

• Compute. tidb-server, Go, Apache-2.0, run from an image. In the target shape every database branch has its own pool of 0 to n tidb-server processes, each started with keyspace-name set to the branch id. A TiDB holds no data, so it can be stopped and started without moving anything. This is the stateless compute idea applied to MySQL.
• Storage. The shared Loams TiKV cluster, the one described in metadata and TiKV. Each branch is one TiKV API v2 keyspace. A keyspace-mode TiDB encodes every key under its keyspace's prefix, so one tenant cannot name another tenant's data from SQL. That is a stronger boundary than table permissions.
• Front end. loams-sqlgate, a Rust MySQL-protocol gate in front of the compute. It terminates TLS, authenticates, finds the branch from the user name or SNI, wakes a suspended pool, and relays packets without parsing SQL.

Everything Loams writes for SQL is Rust: the gate, the control plane (loams-sqldb), the lifecycle machines (loams-sqlrouter), the GC loop (loams-tikv). TiDB, TiKV and PD run as upstream images, with the one TiDB patch described below.

How it worksCopy Anchor Link

The commit and read pathsCopy Anchor Link

A client's COMMIT reaches tidb-server, which runs TiKV's Percolator two-phase commit, with async commit and one-phase commit where it can. The commit is durable once a Raft majority of the region's replicas has the log entry. Neither Loams nor TiDB adds a log of its own: TiKV's Raft log is the write-ahead log (D725). A read asks PD for a start timestamp and pushes coprocessor work down to the regions of the branch's keyspace. A branch never reads another keyspace.

This is a deliberate split of roles. Design §58 §8 records the rule (D1057): TiKV's Raft is the substrate for small transactions and metadata, Loams SQL's data included; loams-wal is the byte log for Postgres WAL; the bucket holds history. A second log in front of TiKV would add latency and no durability.

Keyspace mode, and the one patchCopy Anchor Link

TiDB's classic build accepts keyspace-name, but PingCAP documents keyspaces only for its cloud and next-generation builds, so this is a supported binary path but an unadvertised deployment mode. Loams found one real bug in it. In v8.5.8 the global DDL owner manager campaigns on an etcd key with no keyspace prefix, so every keyspace's TiDB on one PD competes for a single DDL owner, and DDL in every keyspace but the owner's waits forever. Upstream fixed it on master (pingcap/tidb#60403) but not on the 8.5 branch.

Loams cherry-picked that one commit onto the v8.5.8 tag in ostrium-labs/tidb (branch loams/v8.5.8-patches) and builds ghcr.io/ostrium-labs/tidb:v8.5.8-loams.1 from it (SQ1 ruling R5.3). PD, TiKV, BR and TiCDC stay upstream v8.5.8. The patch queue is allowed to hold only small, upstreamable fixes, each dropped when a release carries it.

The rendered TiDB configurationCopy Anchor Link

For the per-branch pools, loams-sqldb renders tidb.toml and a bootstrap SQL file (SQ1 Task 2). The choices that matter:

• split-table = false. TiDB's default gives every table its own region. Measured on the spike stack, a bootstrapped empty database has 61 transactional regions with the default and 1 without it; with the raw region every keyspace gets, that is 2 regions per empty database (R1.4).
• force-init-stats = false, so the port opens without waiting for statistics to load.
• Security Enhanced Mode on, secure bootstrap on and no Unix socket. Root is locked out; the control plane's own user is created by the bootstrap SQL.
• tidb_redact_log = OFF, so clients get MySQL's own error text (Duplicate entry 'x' for key …) rather than redacted values. The slow and general logs, which then carry literals, stay on the pod by default.

The released single TiDB uses a much smaller file, deploy/release/config/tidb.toml, which sets only keyspace-name = "sql_dev". Its comment states the rule that never changes: never run a TiDB without keyspace-name beside Loams, because it would become a second cluster GC worker.

Garbage collectionCopy Anchor Link

On v8.5.8, MVCC garbage collection is cluster-wide: one safe point for every keyspace. Loams' GC loop in loams-tikv is the cluster's GC worker; keyspace-mode TiDBs only run their own delete-ranges (D724). The consequence is honest and shared: one tenant's long transaction, a branch copy or a backup delays GC for everyone. The planned mitigations are transaction-duration caps per compute class and alerts on safe-point lag. Keyspace-level GC exists on PD's master branch but not in a release, and TiKV's side is unmerged, so it waits for an owner decision (Q666).

Suspend and wake through the gateCopy Anchor Link

The gate and the lifecycle machine are built on dev (SQ1 Tasks 3 to 6), though not shipped. A branch moves through RUNNING → SUSPENDING → SUSPENDED → RESUMING → RUNNING. After suspend_after with no command other than COM_PING (5 minutes by default in the design), the gate closes idle sessions and the pool scales to zero. No data moves: the keyspace stays enabled. The next connection makes the gate call EnsureRunning, which starts one TiDB; the gate holds the client for up to 30 seconds, then answers error 1040 (database is resuming, retry).

Two more pieces make the gate usable as a front door. Authentication is caching_sha2_password over TLS only, with Argon2id hashes, per-source and per-user failure limits, and no empty passwords. And idle sessions can move between TiDB members with TiDB's own SHOW SESSION_STATES and SET SESSION_STATES, so a scale-in or upgrade does not have to drop a quiet connection. TiDB refuses to move what it cannot (an open transaction, a cursor, table locks), and the gate retries after the session's next command.

The numbers above are from a shared desktop host, not a production topology. The design's target for resume is a p95 of at most 5 seconds; Kubernetes scheduling is measured later (SQ1 Task 24).

Branches and point-in-time restore (planned)Copy Anchor Link

Here Loams SQL is honestly not Neon-like yet. TiKV keeps row data on local disks with three Raft replicas, and nothing in open-source TiKV shares immutable files between keyspaces. So the v1 branch is a copy (D727):

• Inside the GC window, the copier reads the parent's whole keyspace at the chosen timestamp and writes it into a new keyspace named by the new branch id. TiDB's metadata keys come along, so the child needs no bootstrap.
• Beyond the GC window, the source is BR: a snapshot plus the log backup, restored into the new keyspace. BR v8.5.8 rewrites keyspace prefixes on restore (from source; a test confirms it in SQ1c).
• A restore always creates a new branch; it never overwrites.

Copy-on-write branches need files that two keyspaces can share, which means an object-storage TiKV engine. Two candidate forks of TiKV (tiered SSTs under RocksDB, or a revived shared-storage engine from TiKV's 2022 cloud-engine branch) and the option of waiting for PingCAP's TiDB X are compared in a spike first, and no engine code is written before the owner chooses (D726, Q662).

Key decisions and whyCopy Anchor Link

The decision log is summarised on the decisions page. The ones that shape Loams SQL:

What was rejected. An earlier MySQL design ran a MySQL fork with storage on the bucket, fronted by Vitess (D301, D302). The first rewrite on 2026-10-08 proposed stock MySQL 8.4 with a binlog-on-TiKV service and Vitess; it lost because it put GPL processes and a new log protocol on the commit path, needed per-shard volumes, and resumed slowly (30 to 60 seconds at 10 GiB). PolarDB-X was studied and dropped for its SSPL-licensed CDC and its end-of-life MySQL 8.0 base. These alternatives are in Appendix A of design §47. Every D720 to D739 entry is still marked Proposed in the log; D260's amendment and the owner's answers to Q658, Q661, Q662 and Q667 are recorded.

What is compatibleCopy Anchor Link

The contract is TiDB v8.5's, and design §47 §13.2 publishes it row by row:

• Protocol and dialect: MySQL 8.0 level. The server greets clients as 8.0.11-TiDB-v8.5.8. Nothing specific to MySQL 8.4 (Q658).
• Transactions: ACID with a Raft-majority commit. REPEATABLE READ is snapshot isolation, so write skew is possible. Pessimistic row locks are the default; there are no gap or next-key locks, so a range SELECT … FOR UPDATE does not block inserts into the range. READ COMMITTED works in pessimistic mode. SERIALIZABLE is refused with an error, not silently downgraded.
• Supported: foreign keys (GA since TiDB v8.5.0, with CASCADE and SET NULL), savepoints, online DDL that is not blocked by open transactions.
• Not supported: stored procedures and functions, triggers, events, UDFs, XA, SPATIAL, FULLTEXT indexes (self-managed), the MySQL replication protocol, and character sets beyond ascii, latin1, binary, utf8, utf8mb4 and gbk.

Clients tested so far are tested against the gate, which is not in 0.1.0: captured handshakes of mysql 8.4.10 (libmysql), Connector/J 9.7.0, mysql2 3.15.3, go-sql-driver/mysql v1.9.3 and libmariadb 3.4.10 against TiDB v8.5.8 are replayed and re-encoded byte for byte in loams-sqlgate's tests, and fuzzed in CI. The ORM suites the design names (Prisma, Django, Rails, Laravel at 98 % of their pass count against TiDB) are SQ1 Task 23 and have not run. On the released direct path you are talking to stock TiDB behaviour, so PingCAP's own client compatibility applies.

How it comparesCopy Anchor Link

All three alternatives below have run in production for years. Loams SQL's first release is one TiDB without the serverless parts. The table compares designs, and the last row says how far along each is.

The honest reading: if you need MySQL fidelity, Aurora or PlanetScale is the better choice today, and PlanetScale's deploy-request workflow has no equivalent here. If you want TiDB itself, managed, with serverless and branching now, TiDB Cloud runs the same engine with years of operation behind it. Loams SQL's case is narrower: TiDB on infrastructure you own, sharing one TiKV cluster and one control plane with your other Loams data, with a published path to scale-to-zero and branches.

What is missing, and what is plannedCopy Anchor Link

From the SQ1 plan (docs/plans/2026-10-08-sq1-loams-sql-tidb.md); Tasks 0 to 6 are done.

• The gate in a release (SQ1a is built; shipping needs SQ1b). Until then the released TiDB has a passwordless root, plain text inside the compose network, and no Loams authentication.
• The control plane and API, loams.sqldb.v1: DatabaseService, BranchService, RoleService, BackupService (SQ1 Tasks 7 to 12), records in the metastore, keyspace lifecycle through PD's keyspace API, roles rendered as TiDB grants (reader, writer, ddl, admin).
• GC correctness for TiDB transactions (SQ1 Task 10): the GC loop must read every TiDB's minimum start timestamp from PD's etcd and write the saved safe point. It does neither today.
• Copy branches (Task 13), the Kubernetes driver (Task 14), BR snapshot and log backup to the bucket with PITR (Tasks 15 to 17).
• CDC into Loams streams through loams-sqlcdc, a Rust consumer of TiKV's CDC stream, because classic TiCDC cannot read keyspaces (Tasks 18 and 19). This also gates Loams SQL's part of HTAP phase 1 (Q922).
• Desktop single-node mode on the local TiKV stack (Tasks 20 and 21).
• Evidence for GA (Tasks 22 to 26): the compatibility contract as tests, the ORM suites, performance and density gates (10,000 databases per cluster is a target, not a measurement), observability and a security review.
• Bottomless storage and copy-on-write branches: the S3 spike (Tasks 31 and 32) and the owner's choice.
• The image is amd64 only. On Apple silicon Docker Desktop runs it emulated.

Try itCopy Anchor Link

The release stack starts TiDB with the rest of Loams. From a checkout, following docs/release/preview.md:

git clone --branch v0.1.0 https://github.com/ostrium-labs/loams.git
cd loams/deploy/release
cp .env.example .env
# set TS_AUTHKEY in .env
docker compose up -d
docker compose ps

TiDB then listens on tidb:4000 on the compose network, and nothing forwards it out of the stack. Its root user starts without a password. Any MySQL client on that network can connect; for example, a throwaway client container (the compose file names its project loams, so the stack's network is loams_default):

docker run --rm -it --network loams_default mysql:8.4 mysql -h tidb -P 4000 -u root

ALTER USER 'root'@'%' IDENTIFIED BY 'choose-a-long-password';
CREATE DATABASE app;
CREATE TABLE app.t (id BIGINT PRIMARY KEY, v VARCHAR(64));
INSERT INTO app.t VALUES (1, 'a'), (2, 'bb');
SELECT * FROM app.t;

To reach it from your own device over the tailnet, set the password first, then add a TCPForward entry for port 4000 (for example "4000": {"TCPForward": "tidb:4000"}) to the sidecar's serve configuration, as deploy/release/tailscale/README.md describes. On a VM, deploy/release/byoc/install.sh starts tidb by default.

SourceCopy Anchor Link

• Design: §47 Loams SQL in production; §20 reactive database on TiKV §9.3 and §10 (keyspace-mode TiDB and GC); §58 data access and HTAP §8.
• Plan: SQ1, Loams SQL on TiDB, with its rulings; measurements in docs/sqldb/performance.md.
• Crates: loams-sqldb (control plane, rendering, runtimes, lifecycle host), loams-sqlgate (the MySQL gate), loams-sqlrouter (the Lifecycle machine, checked against a TLA+ spec), loams-tikv (the TiKV client fork and the GC loop).
• Deploy: deploy/release/compose.yaml (the tidb service), deploy/release/config/tidb.toml, deploy/sqldb/spike.
• Decisions: the decisions index.