Consistency tokens
How to read your own writes in Loams, in one process and across processes, with the Loams-Consistency-Token header.
Every write to Loams goes to the log first and is acknowledged with a consistency token: the offsets a reader needs to see to include that write. Indexes catch up in the background, and reads close the gap by merging the log's unindexed tail.
The default: strong reads
A read is strong by default: it sees every write acknowledged before it began. In one process you need to do nothing. Write, then query, and the query sees the write.
An eventual read skips the tail for lower latency, and may miss the newest writes.
Across processes: pass the token
When one process writes and another reads (a worker and an API server, or a job and a webhook), carry the token from the write to the read. It comes back two ways:
-
in the write's response body:
{ "base_offset": 41, "last_offset": 42, "token": [{ "stream": 1, "partition": 0, "offset": 42 }] } -
in the
Loams-Consistency-Tokenresponse header.
Pass it as the Loams-Consistency-Token request header on a read of any object derived from that stream, on the REST or the Connect API, and the read is guaranteed to reflect it:
TOKEN=$(curl -si 127.0.0.1:8080/v1/namespaces/demo/collections/kb/documents \
-H 'content-type: application/json' \
-d '{"ops": [{"upsert": {"id": 4, "source": {"body": "returns", "tenant": "a"}, "vectors": {"embedding": [0.5, 0.5, 0.0]}}}]}' \
| tr -d '\r' | sed -n 's/^[Ll]oams-[Cc]onsistency-[Tt]oken: //p')
curl -s 127.0.0.1:8080/v1/namespaces/demo/query -H 'content-type: application/json' \
-H "Loams-Consistency-Token: $TOKEN" \
-d '{"from": "collections.kb", "retrieve": [{"text": {"field": "body", "query": "returns", "k": 10}}], "limit": 3}'In the SDKs
The SDK runtimes carry a token store. In Python, Loams(..., session_consistency=True) holds a session token across calls, which gives read-your-writes across the calls of one client. Off (the default), every read is strong on its own.
The guarantees
| Scope | Guarantee |
|---|---|
| Within a stream partition | Total order; acknowledged writes are durable |
| Single-object reads (default) | Strong, through the tail merge |
| Cross-object reads | A snapshot per object; with a token, every object derived from those streams reflects its offsets |
| Multi-record writes | Atomic per request within one stream |
| Not provided | Multi-object serializable transactions; interactive OLTP transactions (use Loams Live or Loams SQL) |
Read more in Consistency tokens and The log and consistency.