Loams House

Loams House is an analytics service that speaks ClickHouse. It serves the ClickHouse HTTP interface, and its SQL engine is ClickHouse itself, embedded as chDB. It does not run ClickHouse storage. A House table is an Apache Iceberg table: Parquet data files and Iceberg metadata in your bucket. Statements run in short-lived worker processes, each sealed in an OS sandbox, and the workers can read the bucket only through the House's own read proxy. The House runs in its own binary, loams-fabric, on its own network, next to the Loams engine rather than inside it.

It is for teams that want ClickHouse SQL (its functions, its formats, ReplacingMergeTree-style dedup) over data they own in open files, run on their own tailnet. Use it to explore event and log data with ClickHouse queries from a ClickHouse HTTP client, with the tables kept in an open format rather than in ClickHouse's own storage. Do not use it yet for production dashboards or high-volume ingest. It is experimental. The native ClickHouse protocol, replication, autoscaling and the client and driver test matrix are not built. One statement writes at most 1 GiB. If you need ClickHouse in production today, run ClickHouse or ClickHouse Cloud (see the comparison below).

StatusCopy Anchor Link

Loams House is Experimental in Loams 0.1.0, the current release. It ships in the loams-fabric image and starts with --profile house, but its surface may still change or be reworked. The 0.1.0 feature table describes it like this: "--profile house serves its ClickHouse HTTP interface on your tailnet through its own sidecar, with chDB access control, statement checks and sealed workers. With LOAMS_HOUSE_ENDPOINT the engine reaches it over TLS on a private network; lake tables take INSERT (every declared format, VALUES, SELECT), CREATE TABLE … AS SELECT, TRUNCATE and OPTIMIZE TABLE."

Without LOAMS_HOUSE_ENDPOINT, loams.house.v1 and the console's House page answer house_not_configured. That was the only behaviour in preview.1 and preview.2, where the engine could not reach the House at all.

ArchitectureCopy Anchor Link

The pieces, all in the fabric/ workspace:

• loams-fabric is the binary. Its only role today is house (loams-fabric house).
• loams-house, the front, serves the declared ClickHouse surface over HTTP. Its plaintext listener, port 8123, is bound to loopback only. Its TLS listener on 8443 is the engine's link. It parses and checks statements, manages sessions and settings, admits work into a pool of workers, owns the Iceberg catalog and does every write to the bucket. It links no libchdb.
• loams-house-worker is the only binary that links libchdb. The front spawns it with an empty environment. It seals itself, boots chDB, binds to one namespace for its whole life and runs statements sent as framed hsw1 messages over a socket. A statement is cancelled by killing the worker.
• house-cache (loams-house-cache) is the House's S3 read proxy. It serves ranged GETs, HEADs and, when granted, listings to workers. Each worker may read only its own namespace's prefix. Reads go through a RAM and NVMe range cache with a CRC32C check on every hit. Upstream requests are signed with the front's credentials, which never reach a worker.
• loams-house-lake handles Iceberg metadata, Parquet writing and snapshot commits, on iceberg-rust.
• loams-chdb and loams-chdb-sys are Loams' own FFI over chDB's C header. loams-chdb-sys is the only crate with unsafe.

In the compose stack, house runs on a network named house that no other service joins, plus house-link, an internal network shared only with the engine. house-tailscale shares the House's network namespace, joins your tailnet as loams-house, and proxies HTTPS on port 443 to the front's loopback listener. The House is reachable only through that sidecar and the engine's link.

How it worksCopy Anchor Link

The ClickHouse surfaceCopy Anchor Link

The House serves a declared surface, chsurface-1.0: a versioned list of statements, engines, formats, settings, functions, table functions, system tables and error codes. "Compatible" means the declared surface's tests pass, with deviations listed (D347). On HTTP, the front accepts the statement in ?query=, in the POST body, or both. Credentials can come as X-ClickHouse-User/X-ClickHouse-Key, HTTP Basic or user/password parameters, and responses can be gzip, deflate or zstd compressed. GET /ping answers Ok.. Connect and gRPC-Web for loams.house.v1 are served on the same port.

chDB is pinned to chdb-core 26.9.0, which reports itself as ClickHouse 26.9.2.1. The library is downloaded by SHA-256 at build time and linked dynamically into the worker only.

Three security layersCopy Anchor Link

A House statement is untrusted ClickHouse SQL, and ClickHouse can reach a lot: table functions read files and URLs and connect to other databases. So the House stacks three layers (D774; HS1 ruling R-ACL.1):

• L2, chDB's own access control, is the primary control (R-ACL.1, owner, 2026-10-12). The users workers run as have no SOURCES grants and a constrained settings profile. A table function or engine that reaches files, URLs or other databases is refused by ClickHouse itself. User connections are read-only for DDL and inserts into tables. Settings outside the allow-list are pinned, so a SETTINGS clause that changes them is refused.
• L1, the front's statement checks, stay as a second line. They run before a worker sees the text: a classifier and a deny list over the parsed statement refuse INTO OUTFILE, FROM INFILE, CREATE FUNCTION, dictionary sources, host functions and similar with ClickHouse error 344.
• L3, the OS sandbox, is the boundary for whatever both miss. In pods mode (the default), each worker applies Landlock and seccomp. Landlock allows reads of the code, writes to the worker's own directory, and TCP only to its read proxy. Seccomp allows no execve, no new processes, no namespaces, and IP sockets only. This mode needs no privilege, so it works under Docker's and Podman's default profiles, but it needs Linux 6.7 or later. netns mode adds a user and network namespace per worker with only a loopback. A worker whose sandbox cannot be put in force refuses to boot, and the House stops rather than run unsealed.

The corpus in conformance/clickhouse/corpus fixes these behaviours. classify/ holds statements with the class the front must assign, including adversarial ones. A test checks the front's routing against ClickHouse's own classifier. deny/ holds statements that must be refused, or must pass, with the expected error code. Its test also asserts that nothing connected to a trap URL.

Lake tables on IcebergCopy Anchor Link

A House table is an Iceberg v2 table (v3 when a column needs nanoseconds). Its files live at ns/<id>/house/<db>/<table>-<uuid> under your bucket prefix. A ClickHouse database maps to an Iceberg namespace, ns_<id>.<db>. The catalog is an Iceberg REST catalog in a cluster, or SQLite in --single-node mode, which is what the compose stack runs.

Reads pin each table to its current metadata file and read it through views over chDB's Iceberg reader, pointed at house-cache. Time travel uses loams_snapshot_id or loams_as_of. chDB's own filesystem cache is off. The House's cache is the bounded, shared house-cache (D766).

Writes: INSERT commits an Iceberg snapshotCopy Anchor Link

INSERT takes every declared input format, plus VALUES and INSERT … SELECT. CREATE TABLE … AS SELECT and TRUNCATE work too. The path:

1. The worker loads the rows into a temporary Memory table and casts and orders them into the table's types. It returns them as Arrow. The worker holds no bucket grant and no credentials.
2. The front writes Parquet data files with Iceberg column statistics and uploads them.
3. The front commits. It writes a new metadata file that must not exist yet (files are never rewritten), then swaps the table's metadata pointer by compare-and-set. Each table has one commit queue that makes at most one append snapshot per commit interval (default 1,000 ms), so concurrent inserts share a commit.
4. The client is answered only after the commit.

If another commit got there first, the front reloads the table, decides deduplication again, rebuilds the append and retries, up to 20 attempts with jitter. A retry after an ambiguous failure first looks for its own commit ID, so it cannot apply twice. Inserts are deduplicated by insert_deduplication_token, or by a hash of the data, over a window of recent commits. TRUNCATE commits a delete snapshot and keeps history. A test runs 2 fronts with 16 writers each, 50 inserts per writer, and checks that every commit lands.

chDB can write Iceberg itself, but the House does not use it. HS1's spike found that it wrote metadata without a catalog or compare-and-swap, and that its OPTIMIZE deleted the table's earlier files (ruling R1.5).

CompactionCopy Anchor Link

Replacing and Summing tables merge rows the way ClickHouse's FINAL does. OPTIMIZE TABLE … [FINAL] [DEDUPLICATE] and a background scheduler rewrite a table's files and commit an Iceberg replace snapshot that requires the table's main branch to be unchanged. A concurrent commit always wins, and the compaction is abandoned. OPTIMIZE retries 3 times, then answers error 210. The scheduler checks every 30 s and compacts when a partition has 16 or more files under 32 MiB, or when 20 % or more of its rows are unmerged. Compaction is whole-table for now.

The engine linkCopy Anchor Link

With LOAMS_HOUSE_ENDPOINT=https://house:8443, the engine proxies loams.house.v1 to the House over TLS on house-link. It trusts only the certificate the House made on first start and signs in with a link key, after its own auth has let the caller in, naming the caller's Loams principal. The House gives each principal the scopes HOUSE_PRINCIPALS lists (read, write, ddl, admin). A principal that is not listed gets no House access. Every statement is recorded with its principal in the House's audit log. This is what lets the console's Analytics page run queries.

Key decisionsCopy Anchor Link

The full log is in the decisions index.

Why D45 was reversed, and only for the House. D45 was right about storage: data in Loams is already open Iceberg, and a second ClickHouse server with its own disks would duplicate it. But it threw away the ClickHouse dialect along with ClickHouse storage, and that dialect is what ClickHouse-native tools and apps speak. D347 keeps D45's storage half and reverses its interface half. The House speaks ClickHouse over a declared, tested surface. Every table stays Iceberg. The engine binary stays free of ClickHouse.

No ClickHouse fork. Loams forks Grafeo for Loams Graph, but it does not fork ClickHouse or chDB. It pins upstream libchdb by version and checksum. The House's risk is how much ClickHouse can do, not bugs in a small codebase, so the answer is configuration and isolation, not patches: chDB's own grants and settings profile (R-ACL.1), the statement checks, and the sandbox.

What is compatibleCopy Anchor Link

• Protocol: ClickHouse HTTP (HTTP/1.1), over HTTPS on port 443 through the tailnet sidecar. 0.1.0's guide: "Any ClickHouse HTTP client works the same way (port 443, TLS, user default)." The native TCP protocol (9000/9440) is not served.
• SQL: ClickHouse 26.9 semantics through chDB, within chsurface-1.0. Table functions that reach outside (file, url, s3, remote, mysql, postgresql, executable and others) are refused by design.
• Tables: Iceberg v2 with Parquet data files. The format is open, but in the compose stack the catalog is the House's own SQLite one; an Iceberg REST catalog for outside engines (DuckDB, Spark, Trino, ClickHouse) is a planned row in the 0.1.0 feature table.
• Clients tested: no ClickHouse driver is in the House test suite yet. The tests use plain HTTP and the generated loams.house.v1 Connect client. Where it is installed, clickhouse local 26.7 serves as an optional reference for compaction and materialized-view results. The planned driver matrix (clickhouse-client, clickhouse-connect, clickhouse-go v2, clickhouse-java JDBC v2, clickhouse-rs, Grafana, Metabase and others) is HS1 Task 33.
• Console: the Analytics page (web/plugins/house) has a ClickHouse SQL editor, a results grid with ClickHouse types, progress and cancel, the plan view and query history. It is read-only by default and asks before a write.

Compared with the alternativesCopy Anchor Link

ClickHouse is far more complete. It has the native protocol, every engine and table function, replication, sharding, materialized views on MergeTree, and years of production use. ClickHouse Cloud adds a managed service on object storage. If you need any of that today, use them.

The House makes sense when you want ClickHouse queries over open Iceberg files that you own, on your own tailnet, with statements isolated per process, next to the rest of a Loams stack. DuckDB is the simpler choice when one process and DuckDB's dialect are enough. MotherDuck is its managed counterpart. Neither speaks ClickHouse SQL.

What is missing todayCopy Anchor Link

From the HS1 plan:

• Realtime tables (Task 14), which wait on the Fabric foundation (FL1).
• Autoscaling and scale to zero (Task 24); HA and disaster recovery (Task 25); packaging and platforms, including a Helm chart (Task 26); upgrades (Task 27).
• chsurface-2.0 and its harness (Task 29); the native protocol codec and server (Tasks 30, 31); ClickBench, TPC-H and the upstream stateless test subset (Task 32).
• The driver and BI matrix (Task 33), Arrow Flight SQL for the House (Task 33a) and the ClickHouse ADBC driver as a client (Task 33b).
• Performance gates, fault evidence, a security review and fuzzing, docs, and the GA flip (Tasks 34 to 38).
• Loams identities in the House: principals are operator config (HOUSE_PRINCIPALS) until the multi-tenant work.
• Limits: one statement writes at most 1 GiB; compaction is whole-table; writes to Iceberg v3 tables are refused; Iggy pipes are deferred (Task 18).
• Iceberg analytics through a REST catalog for outside engines is a separate planned row in the 0.1.0 feature table.

The House stays labelled beta in its own design (D760) until every item on the HS1 exit checklist is ticked.

Try itCopy Anchor Link

From the 0.1.0 release guide. docker compose --profile house up -d starts the House and its sidecar. Give the sidecar its own auth key in HOUSE_TS_AUTHKEY, or leave it empty to reuse a reusable TS_AUTHKEY. Then, from a device on the tailnet:

docker compose exec house cat /var/lib/loams-house/house.key    # the password, made on first start
curl -u "default:$PASSWORD" https://loams-house.<tailnet>.ts.net/ --data-binary 'SELECT version()'

To wire the engine to it, so loams.house.v1 and the console's Analytics page work, follow deploy/release/README.md:

echo 'LOAMS_HOUSE_ENDPOINT=https://house:8443' >> .env          # wire the engine to it
echo 'HOUSE_PRINCIPALS=tailscale:you@example.com=read+write+ddl' >> .env   # who may use it
docker compose --profile house up -d house house-tailscale loams
docker compose exec house cat /var/lib/loams-house/house.key   # user `default`'s password
curl -u "default:$PASSWORD" https://loams-house.<tailnet>.ts.net/ --data-binary 'SELECT 1'

On your own VMs (BYOC), add the House with --services "loams tailscale tidb house house-tailscale". pods mode needs Linux 6.7 or later on the host; on an older kernel the workers refuse to boot.

SourceCopy Anchor Link

• Crates: fabric/crates/loams-fabric, loams-house, loams-house-worker, loams-house-cache, loams-house-lake, loams-chdb.
• Conformance: conformance/clickhouse.
• Deploy: deploy/release/compose.yaml.
• Design: 49, Loams House in production; 32, Loams Flow, Event Fabric and House; 08, Analytics (Iceberg).
• Plans: HS1, House in production; the HS1 spike; FL1, Fabric foundation; FL2, House SQL; the FL2 chDB dependency spike.