First collection and hybrid search
Create a collection over the native REST API, write documents with vectors, and run one query that fuses vector kNN, BM25 and a filter.
With loams dev running, everything below uses the native REST API on port 8080. These examples use POSIX shell quoting; on Windows, use WSL or adapt the quoting for PowerShell and call curl.exe.
1. Create a collection
A collection is documents with a schema: Lance holds the vectors and columns, Tantivy the full-text index, and one manifest ties them together. This one has a text field, a keyword field for filtering, and a 3-dimensional vector.
curl -s 127.0.0.1:8080/v1/namespaces/demo/collections -H 'content-type: application/json' -d '{
"name": "kb",
"schema": {
"fields": [
{"name": "body", "source_path": "body", "kind": {"text": {"analyzer": "standard", "positions": true}}, "indexed": true, "fast": false},
{"name": "tenant", "source_path": "tenant", "kind": "keyword", "indexed": true, "fast": true}
],
"vectors": [{"name": "embedding", "dim": 3, "distance": "cosine"}],
"sparse_vectors": [], "dynamic": "ignore", "max_fields": 1000
}
}'demo is the namespace, the unit that groups streams and collections.
2. Write documents
Writes are a batch of operations. Each upsert carries the document's source and its vectors.
curl -s 127.0.0.1:8080/v1/namespaces/demo/collections/kb/documents -H 'content-type: application/json' -d '{"ops": [
{"upsert": {"id": 1, "source": {"body": "refund policy", "tenant": "a"}, "vectors": {"embedding": [1.0, 0.0, 0.0]}}},
{"upsert": {"id": 2, "source": {"body": "shipping times", "tenant": "a"}, "vectors": {"embedding": [0.9, 0.1, 0.0]}}},
{"upsert": {"id": 3, "source": {"body": "refund window", "tenant": "b"}, "vectors": {"embedding": [0.0, 0.0, 1.0]}}}
]}'The response carries a consistency token, and so does the Loams-Consistency-Token response header. Reads are strong by default, so the next query sees these documents at once. See Consistency tokens for reads from other processes.
3. Run a hybrid query
One request retrieves by vector and by BM25 text match, filters on tenant, and fuses the two rankings with reciprocal rank fusion (RRF):
curl -s 127.0.0.1:8080/v1/namespaces/demo/query -H 'content-type: application/json' -d '{
"from": "collections.kb",
"retrieve": [
{"vector": {"field": "embedding", "query": [1.0, 0.0, 0.0], "k": 10}},
{"text": {"field": "body", "query": "refund", "k": 10}}
],
"filter": {"term": {"tenant": "a"}},
"fuse": {"method": "rrf", "k": 60},
"limit": 3
}'Document 3 matches refund too, but the filter keeps only tenant a, so it is not in the results.
4. The same query in SQL
The SQL table functions vector_search, text_search and rrf express the same plan:
curl -s 127.0.0.1:8080/v1/namespaces/demo/sql -H 'content-type: application/json' -d @- <<'JSON'
{"query": "SELECT _id, body, _score FROM rrf(vector_search('kb', [1.0, 0.0, 0.0], 'embedding', 10), text_search('kb', 'refund', 'body', 10)) WHERE tenant = 'a' LIMIT 3"}
JSONThe same SQL runs over Flight SQL and the Postgres and MySQL wire listeners.
5. Read the bucket directly (optional)
POST /v1/namespaces/{ns}/collections/{c}/scan returns a pinned Lance version, which pylance, Ray, Polars or PyTorch read straight from the bucket (pip install pylance==12.0.0):
import json, urllib.request
import lance
request = urllib.request.Request(
"http://127.0.0.1:8080/v1/namespaces/demo/collections/kb/scan",
data=b"{}", headers={"content-type": "application/json"},
)
plan = json.load(urllib.request.urlopen(request))
if plan["lance"] is not None: # null until the first commit
dataset = lance.dataset(plan["lance"]["uri"], version=plan["lance"]["version"])
print(dataset.count_rows(), "live documents; tail:", plan["tail_records"], "records")Writes the pinned version does not hold yet (the plan's tail) are readable through the native API or Flight SQL.
Where to go next
- Call the same engine from your language: SDKs.
- Every route and field: the HTTP API reference.
- Why the query is shaped this way: Hybrid retrieval.