Beside Loams, not inside it: Sail, RisingWave, Neon and MySQL

Loams has one firm rule about other engines: it integrates with them over protocols and open formats, and never links them into its binary. Engines built on DataFusion or Arrow move in version lockstep, and Loams’ versions are pinned by Lance; some engines embed a Python interpreter or need a JVM; some carry licenses Loams does not link against. Running them as separate processes keeps Loams’ binary, license and upgrade cadence its own.

This post covers four such engines. Sail and RisingWave remain future integrations. Loams Postgres and Loams SQL ship in Loams 0.1.0, both as separate processes in the compose stack; their serverless parts (the PgDog front door, the Loams SQL gate, scale to zero for MySQL, branches through the control plane for MySQL) are still being integrated. The original spike findings below retain their historical context.

Sail: Spark Connect in RustCopy Anchor Link

What it is. Sail (v0.7.1) is a drop-in replacement for Spark's compute, written in Rust on Arrow and DataFusion. It implements the Spark Connect protocol, the gRPC protocol with which PySpark 3.5 and later can talk to a remote Spark server, so PySpark clients connect to Sail with SparkSession.builder.remote("sc://…") and do not know the difference. It runs locally or in a Kubernetes cluster mode (a server, a driver per session that launches worker pods, and shuffle through object storage since 0.7), and runs Python UDFs in an embedded CPython.

What it covers. Spark SQL and the DataFrame API; Python UDFs, pandas UDFs, UDTFs and the mapInPandas family; Iceberg v1 to v3 with copy-on-write writes (merge-on-read writes and deletion-vector reads are still in progress, so tables that use deletion vectors need checking before Sail reads them), through REST, Glue, Unity and Hive catalogs; S3, GCS, ADLS and HDFS.

What it does not. RDDs and SparkContext (Spark Connect does not carry them at all), Java and Scala UDFs, MLlib and pandas-on-Spark; Structured Streaming is not ready. Sail's own CI shows about 91% of Spark 3.5's Connect tests passing and about 71% of Spark 4.2's. Its performance claims are its own, derived from TPC-H runs by the vendor, and its compatibility checker does not verify behavioural parity.

For Loams. Two roles. Sail is a named reader in the Iceberg exit gate, so PySpark curation jobs on stateless compute read Loams’ keyed tables correctly. And in the jobs proposal (platform part 6), Loams runs one Sail server per namespace behind an authenticating sc:// proxy, with Apache Spark on Kubernetes as the fallback for jobs Sail cannot run. Sail's crates are on DataFusion 55 and arrow 59 against Loams’ 54 and 58, and its maintainers declined Rust-level Lance integration for the same reason, so Sail is always a separate process.

RisingWave: streaming SQL on object storageCopy Anchor Link

What it is. RisingWave (v3.1.0) is a streaming database: you define sources, materialized views and sinks in Postgres-dialect SQL, and it maintains the views incrementally as data arrives. It is written in Rust but is a service, not a library: a Postgres-wire frontend, a meta service, compute nodes and compactors. Its state lives in Hummock, an LSM-tree store on object storage, with epoch-based checkpoints. It reads from Kafka, Pulsar, Kinesis and NATS, and reads and writes Iceberg, including through Lakekeeper.

For Loams. RisingWave is Loams’ companion stream processor: it keeps Loams out of stateful stream processing (windowed joins with checkpointed state) while giving users joins, windows and materialized views. Before Loams streams speak Kafka, RisingWave can write into Loams through its Elasticsearch sink (Loams’ _bulk subset), its HTTP sink and, later, its Iceberg sink; once they do, it reads Loams streams directly. (Since this post, D408 deferred a Kafka gateway inside Loams in favour of Apache Iggy's Kafka gateway; Kafka wire is still planned in 0.1.0.) In the jobs proposal it is also where Flink SQL jobs move, after a dialect port, since RisingWave does not speak Flink SQL.

Limits. Some features are "Premium" behind a license key, with a free tier capped at a small size; Loams would use none of them (the Iceberg REST catalog is not Premium; the Glue catalog is). It does not replace Flink's DataStream API or its long tail of connectors.

Neon: Postgres whose storage is the bucketCopy Anchor Link

What it is. Neon splits Postgres in three:

• Compute: stateless Postgres with a modified storage manager that asks for pages over the network instead of reading local files.
• Safekeepers: a Paxos-replicated service that accepts the compute's WAL, so a commit is durable once a quorum has it, and offloads WAL to object storage.
• Pageserver: ingests the WAL, stores page versions as immutable layer files in object storage, and serves any page as of any WAL position. Its local disk is a cache.

A storage broker connects them, and a storage controller handles placement and failover. Because every page is versioned, a branch is a new timeline whose ancestor is an existing one at a chosen WAL position: copy-on-write, with no data copied.

For Loams. Our spike ran it on RustFS: branches in 36 ms, isolated both ways, logical replication whose slot survived a compute replacement, OpenFGA's and GlitchTip's migrations unchanged, and full recovery from the bucket after the pageserver's disk was wiped. The proposal has Loams act as Neon's control plane, route Postgres connections by database name, give every agent workspace its own branch, and feed changes into collections. In 0.1.0 the control plane ships as pg-control and pg-runner; routing by name and change feeds into collections are still to come. Platform part 7 and the Loams Postgres page have the details.

Historical evaluation. The public repository has been nearly inactive since August 2025, after the Databricks acquisition; its Postgres is 16.9 and 17.5 from May 2025 with Neon's patches; the control plane and the proxy's authentication backend were never open source. Those were maintenance concerns recorded in the original evaluation. The repository now implements a Neon-compatible WAL service and router verification tooling, with deployment/benchmark harnesses. Those concerns decided the ownership question: D231 (2026-09-29) made Loams Postgres a fork of Neon, and D820 (2026-10-09) made it a hard fork, ostrium-labs/loams-postgres, that no longer follows upstream, so releases, Postgres patches and security fixes are ours. Loams 0.1.0 runs the fork's own signed release, lp-2026.10.1. Key decisions lists the forks and reversals; see also the updated SQL inventory.

MySQL: Loams SQL on TiDB and TiKVCopy Anchor Link

Updated 2026-10-17. When this post was first published, this section evaluated a GPL MySQL distribution whose LSM storage engine kept data, WAL and binlog in object storage. Our spike found that its storage engine had no foreign keys (so Forgejo's migrations failed), that ENGINE=InnoDB was silently rewritten, and that commits after the last object-store snapshot could be lost when a node died with its local volume. That direction has been replaced.

What it is now. MySQL on Loams is Loams SQL: MySQL-compatible SQL served by a lightly patched TiDB (v8.5.8-loams.1, D721), with each database in its own TiKV keyspace. TiKV is already Loams' transactional store (it backs Loams Live and, on clusters, metadata), so Loams SQL adds a SQL layer rather than a second storage system, and keyspaces give every tenant or agent workspace an isolated database on shared infrastructure. TiDB and TiKV are Apache-2.0, and both run as separate processes, reached over the MySQL wire protocol.

Current scope. Loams SQL is available in 0.1.0 as one keyspace-mode TiDB on the stack's TiKV, reachable inside the compose network; provisioning and branches through the control plane, its gate and scale to zero remain an integration track. The Loams SQL page tracks what is available.

What the four have in commonCopy Anchor Link

Each one does a job Loams deliberately does not: Spark's API, stateful stream processing, Postgres OLTP, MySQL OLTP. Each keeps its durable state in object storage or in TiKV, which is why they fit next to a bucket-native platform. And each is a separate process with its own license, reached over Spark Connect, the Postgres or MySQL wire protocol, Iceberg or Loams’ own APIs.

The last post in this series covers the smaller libraries inside Loams that carry more weight than their size suggests.