On this page
There is a moment in every database evaluation when someone asks: "how much of our code do we have to change?" If the answer is "all of the retrieval layer, before you know whether this works", most evaluations end there. They should.
Our position is that a new data system should earn trust through the clients you already run. Your LangChain app talks to Qdrant through qdrant-client. Your search code talks to Elasticsearch through elasticsearch-py. Your services talk to Postgres through a driver and an ORM. Loams speaks those protocols, so the first test is changing a URL. Our own native API comes second, for the things those protocols cannot express.
The other half of the position is less popular: compatibility is a scope set by tests, not a promise of parity. We do not claim to be Elasticsearch. We claim that a named list of test suites passes unmodified, and we refuse everything else loudly. This post explains both halves and what they cost.
What Loams speaks
The engine's protocol footprint was narrowed on purpose in the architecture review (D42): the native REST and gRPC API, Arrow Flight SQL, the Qdrant REST and gRPC API, and a targeted Elasticsearch subset. The rationale in the decision log is one line we still agree with: "Emulating five products' protocols is a five-product trap." Qdrant and Elasticsearch carry the AI-framework adoption, Flight SQL is the clean path for every language, and everything else should be reachable through open formats.
Around that core, other products add their own wire protocols, each for a reason:
- Postgres and MySQL wire over collections, so a SQL client or BI tool can read collections. Both ship in 0.1.0, off unless you turn them on.
- Loams Postgres and Loams SQL are real databases (a Neon fork and a patched TiDB), so their wire protocol is not a translation at all: it is the product. For Postgres we made that a rule (D1030): Postgres wire is the SQL protocol, and new capabilities arrive as opt-in extensions and APIs around it, never as a new wire protocol.
- The House serves a declared ClickHouse HTTP surface,
chsurface-1(D347), so ClickHouse-native applications run unchanged against it. - Loams Durable speaks the Resonate protocol, so agent code uses the Resonate SDKs unmodified.
Why not just a better API
We have a native API, and it is better for what it was designed for. One request does vector search, BM25, filters and fusion as one planned query (the previous post walks through it), carries a consistency token and returns a performance block. Neither the Qdrant API nor the Elasticsearch DSL can say "seed with both, fuse, then expand two hops in the graph". So why spend engineering on other people's protocols?
Because adoption runs through frameworks. The application code that talks to a vector store today is rarely hand-written. It is LangChain's or LlamaIndex's vector-store class, which takes a qdrant-client or an elasticsearch-py instance. Supporting those clients means every framework integration built on them works on day one, without a Loams plugin in each framework's repository.
Because the way out matters as much as the way in. If Loams speaks Qdrant's API and you decide Loams is not for you, you point the same client back at Qdrant. A system that only speaks its own API makes leaving as expensive as arriving, and careful teams price that in.
Because the comparison becomes honest. With the same client and the same test suite against both systems, "does Loams give the same answers as Elasticsearch" is a test run, not a sales conversation. That is how our gates work: the BEIR relevance gate compares rankings with Elasticsearch's BM25, and the Qdrant suites check results against brute force with Qdrant's own formulas.
Scoped by tests, not by feature lists
Here is the rule that keeps compatibility affordable (D13): the compatibility scope of each protocol is defined by external conformance suites, the client libraries' and frameworks' own tests, run unmodified. If a suite we gate on sends it, we implement it. If no gated suite sends it, it waits, however popular it is on Stack Overflow.
For Elasticsearch this rule produced Phase A (D48): exactly what the LangChain and LlamaIndex Elasticsearch suites and the BEIR harness send. Document APIs and _bulk, _search with the core Query DSL, knn, hybrid search with RRF, _delete_by_query, and minimal index admin. _msearch is in, because BEIR batches every query through it. Aggregations, point in time and highlighting are out until a gated test needs them, because none of those suites sends them. An earlier draft gated on elasticsearch-py's own client suite; we dropped that (D48 replaced D32) when it turned out its test fixture needs snapshot, data-stream and template endpoints that the framework suites never exercise.
The same rule pulled work forward when the tests asked for it. Qdrant's sparse vectors were planned for a later phase. The LangChain and LlamaIndex Qdrant gates include 172 sparse and hybrid server tests, so sparse vectors moved into M1 (D31).
What this looks like in CI today, from the release's feature table:
| Surface | Tested with |
|---|---|
| Qdrant REST and gRPC | qdrant-client 1.15.1 over REST with the legacy methods, and 1.19.1 with the query API over REST and gRPC; each result checked against brute force with Qdrant's formulas |
| Elasticsearch subset | elasticsearch-py 8.19, through the LangChain and LlamaIndex suites, and BEIR |
| Arrow Flight SQL | The ADBC Flight SQL drivers for Python and Go are an M1 exit gate (D49) |
The rule has a corollary we care about: outside the scope, fail loudly. A Qdrant request for a feature we do not implement (multivectors, geo conditions, order_by, formula and a long list more) gets HTTP 501 with Unsupported in Loams: <feature>, or gRPC UNIMPLEMENTED. A silently ignored parameter is how compatibility layers lose data and trust. A refused one is a bug report you can act on.
The same rule, applied to graphs, analytics and SQL
The test-suite rule shaped what we refused as much as what we built.
No Bolt. We dropped Neo4j's Bolt protocol, the Cypher subset and the procedure shims (D44). GraphRAG needs vector or BM25 seeds, one or two hops and a rerank, planned together, and Cypher's parser and recursive execution have a fraction of the demand. When Loams Graph arrived, its surface became GQL, the ISO standard (D634), as a declared, versioned profile gql-1 listed by ISO feature id, with undeclared features answering UNIMPLEMENTED (D747). Then the frameworks pushed back: LangChain and LlamaIndex generate Cypher. So an opt-in Cypher dialect, off by default, per graph, gated by an openCypher TCK subset and still without Bolt or APOC, is proposed (D748). That is the rule working: the demand is measured by what the tools send.
No ClickHouse in the engine, then a declared one in the House. The architecture review dropped the ClickHouse HTTP interface and dialect from the engine, because Iceberg tables already give external engines SQL access (D45). A week later we reversed that for the House only (D347): it serves chsurface-1, a versioned list of statements, engines, formats, settings, functions and system tables, where "100% compatible" means 100% of the declared surface's tests pass and every deviation is on an allowlist. The engine still speaks no ClickHouse.
No Kafka broker in the engine. Rebuilding consumer-group rebalancing and transactions is a project of its own (D43). Kafka clients of our Event Fabric use Apache Iggy's Kafka gateway, with our contributions, and a gateway in the engine is deferred (D408).
No SQL Server protocol. We looked hard at TDS, because it does several things better than Postgres wire: in-band cancel, explicit feature acknowledgement, session recovery, server-initiated redirect (design 58 §2). We still said no for now (D1031). Microsoft's specification notice says it "has patents that might cover your implementations" and grants no patent licence by itself. SQL Server tooling wants a SQL Server (its catalog views and procedures), not a wire format. T-SQL semantics are the real cost: Babelfish, open source since 2021, still lists bulk copy, partitioning and most XML as unsupported. And there is no production Rust TDS server to build on. If a paying migration ever needs it, we would port Babelfish onto Loams Postgres rather than write T-SQL in Rust.
Instead, the good ideas from TDS are designed as opt-in Postgres extensions in loams-gateway (D1032–D1038): resumable sessions, a GoAway-and-redirect notice, cancel on the same connection, and request attributes. A client asks for them with _pq_.loams.* startup parameters, the reserved namespace Postgres keeps for protocol extensions. The gateway strips them before anything upstream sees them and acknowledges what it agreed to. A plain client sees stock Postgres. Loams clients will also send a random "grease" parameter that must be refused, so a middlebox that chokes on unknown parameters is found at once, not when the first real extension ships. That is the most compatible way we know to extend a protocol: never require it.
What compatibility costs
None of this is free, and the costs are the reason for the scoping rule.
Divergences are permanent work. Even inside the scope, a translation layer does not behave exactly like the original, and each difference has to be found, decided and written down. A few from the current lists (design 06 §7 and §8):
- Elasticsearch's
_seq_nois Loams' partition offset and_versionis_seq_no + 1, so versions are not dense. - A search across several indices fails as a whole when one index fails; Elasticsearch returns hits from the others with
_shards.failed. - In Qdrant's API, DBSF fusion over Euclid or Manhattan prefetches normalizes Loams' larger-is-better scores, where Qdrant normalizes raw distances.
- Qdrant's
waitflag only changes the status string, because an acknowledged write is already visible in Loams. - Shard, replica, WAL and optimizer settings are stored and echoed, and change nothing.
Every one of these is documented in the crate (loams-es, loams-qdrant) with the ruling that decided it. That list only grows.
Other people's specs are not always usable as written. Qdrant's protobuf definitions are Apache-2.0 and we vendor nine of them. Its REST types we wrote by hand, because the 1.19 OpenAPI document omits the legacy routes the server still serves, and generators get its anyOf precedence wrong (D133). Elasticsearch's REST YAML spec tests would be the obvious conformance source, but Elastic relicensed in 2024, so their licence has to be checked before we vendor any of them. Clients also check identity: the Elasticsearch gateway reports version 8.19.0 and sends X-Elastic-Product: Elasticsearch on every response, because the official client refuses to talk to a server that does not.
Every listener is attack surface. Each protocol is a parser facing the network, with its own authentication story. In 0.1.0, the hosted console reaches only the engine's main port, through your tailnet, under the configured auth mode. The Qdrant, Elasticsearch, Flight SQL, Postgres-wire and MySQL-wire listeners stay inside the compose network, and the release notes say plainly why: the auth modes are specified for the main port, and the other listeners are not all reviewed against them yet. Do not publish them. The gateways bind 127.0.0.1 by default for the same reason (D111).
Compatibility can leak into the core if you let it. The way to lose a codebase is to let Elasticsearch's version semantics, Qdrant's payload rules and Postgres's type system each reach into storage. Our rule is principle 6 of design 01: compatibility is a gateway concern. Each frontend parses its protocol and translates it into a small set of internal logical operations (the search IR, writes to streams, SQL), with one executor per operation. The Qdrant REST and gRPC APIs convert into the same model and run on the same executor. A new protocol adds a frontend; it does not add a storage path.
What we would do differently
Two lessons from getting here.
Write the scope down before writing the code, and pick the right suite. Our first Elasticsearch plan gated on the client library's own test suite, which pulled in endpoints the framework suites never exercise (D32). Tying the scope to the framework suites that real applications run (D48) cut it to what matters. Every compatibility surface since (the Qdrant phases, chsurface-1, gql-1) started as a declared, versioned list.
Count requests, not features. The cheapest compatibility work is the work you can prove nobody needs. The sparse-vector reversal (D31) and the Cypher dialect (D748) both came from looking at what real clients send, and both were the opposite of what we planned.
Where we are
In 0.1.0 the Qdrant API, the Elasticsearch subset, Arrow Flight SQL, and Postgres and MySQL wire over collections are available: built, tested in CI and usable, at 0.x stability. The House's ClickHouse HTTP interface is experimental, behind an opt-in profile. The _pq_.loams.* Postgres extensions and the Cypher dialect are designed, not built.
The most useful thing you can send us is a compatibility report: the client, its version, the call, and what came back. If it is in scope, it is a bug. If it is not, it is evidence for the next phase. Open an issue, and see design 06 and design 58 for the full scope tables.