The guidance corpus has served the underwriting agent for months. The vectors were computed with one embedding model, and every query for UW-004 brings back R-ROOF-03 near the top. Then the team adopts a new version of the model. A release ships the new query encoder on Friday afternoon; the migration job that re-embeds the stored passages is scheduled for Monday.
That gap is the mistake. For the whole weekend, query vectors from the new model are compared with stored vectors from the old one. Both have the same dimension, so nothing errors. The results look plausible and are wrong: this is this site's own case UW-R06, where a query from toy encoder B against an index built with toy encoder A shares one of five neighbors with the correct answer. Nobody notices until an underwriter asks why the agent cited a sprinkler note for a roof question.
The index, the SQL and the model all worked. What failed was an operation on live data: a change rolled out in the wrong order. This page is about those operations, the day-two work of a retrieval system.
What this is not. This page does not teach traces and logs as a discipline; that is observe.sammuthu.com. It does not cover injection defense or access leaks as guardrails in depth; that is guardrails.sammuthu.com.
Retrieval is a query over data that keeps changing#
Every page of this site keeps one primitive true. An embedding column holds one vector per row: a list of numbers, computed by a model from the row's text, that places the row in a space of meaning. A vector query compares the query's vector with the rows' vectors, ranks the rows by distance and returns the nearest k. The model reads only what retrieval returned. Retrieval is a query. The model reads only what it returns.
Operations is what keeps that sentence true while everything under it changes: policy versions are published, precedents revoked, documents removed, model versions retired. Each change touches rows, and each row may have copies: a vector, an index entry, a chunk, a cached answer. If one copy changes and another does not, the query returns rows that no longer exist, belong to someone else, or come from a superseded policy. The model then reads them as if they were true.
Before AI. Database teams solved most of these problems long ago. A schema change without downtime is done in phases often called expand and contract: add the new column, write both, backfill, switch reads, drop the old column later. Deletes have to reach the search index and the cache. Multi-tenant tables need a tenant filter on every query. Retrieval systems inherit all of it, with one new twist: a vector cannot be read or checked by a person, so a wrong one fails silently.
Changing the embedding model without downtime#
The standard first, stated as a rule. A model change must be run as a migration with two sets of vectors side by side: the old vectors keep serving every query until the new ones are complete, evaluated and chosen. Queries must always be encoded by the same model, version and preprocessing as the vectors they are compared with, and the stored embedding_model value must be checked on every query. An embedding migration is that whole sequence: recomputing every stored vector with a new model while the system stays online.
Why not re-embed in place? During the update half the rows would hold old vectors and half new ones, and no query vector can be compared with both. And if the new model turns out worse, there is no way back without re-embedding again. The side-by-side design avoids both problems.
The sequence has seven steps, and each one can be checked before the next:
- Add the new column (or a table keyed by row id and model), empty. Nothing reads it yet.
- Dual write. Every new or changed row now gets both vectors in one transaction, so rows written during the backfill are not missed. Dual write means writing one change to two places.
- Backfill. A job fills the new column for existing rows in batches, selecting rows whose new vector is missing, so it can stop and resume.
- Build the index on the new column, concurrently so writes keep flowing.
- Evaluate. Run the labeled queries against both columns, each with its own query encoder, overall and per slice.
- Cut over. The cutover switches reads to the new column and the new query encoder together, in one configuration change.
- Keep the rollback window open, then drop. The old column stays complete, still dual-written, so rolling back is switching reads back. Only after real traffic is it dropped.
Diagram: An embedding migration: readers see nothing change until one switch
Read left to right. The new column is added, dual-written, backfilled, indexed and evaluated while every query still uses the old column and the old query encoder. The cutover moves the column and the encoder together. The old column stays complete during the rollback window, so going back is a switch. Only then is it dropped. Illustrative of the example's toy encoders; nothing here was measured on a real model.
- Before the cutover: readers unchanged
- Add the new column: A second vector column (or table keyed by row id and model) next to the old one.
- Dual write: New and changed rows are written to both columns in one transaction, so the new column cannot miss them.
- Backfill in batches: Existing rows get their new vector; the job can stop and resume.
- Index and evaluate: Each column is scored with its own query encoder; the new one must meet the bar.
- The cutover: one switch
- Switch column and encoder together: Reads move to the new column and the new query encoder at the same moment.
- After: a rollback that costs nothing
- Rollback window: Going back is switching reads back; no vectors are recomputed.
- Drop the old column: After the drop, a rollback would need a full re-embedding.
The diagram shows what makes the design safe. Until the cutover, nothing a reader sees has changed; after it, the old column is still complete, so the way back is a switch, not a second migration. And the query encoder moves only at the cutover, with the column it reads: the order the Friday release got wrong.
In the insurance build. In that build, a model change is planned as a new column, a backfill and a switch of reads. Rows record an embedding_version, so stale rows can be found and re-embedded in batches. That is the design as that build states it; this page does not report a model change carried out in it.
The same steps in SQL#
On PostgreSQL with pgvector the steps map to ordinary statements. The block below is a documented shape assembled from pgvector's README (read on 2026-11-18); it was not run here. It shows what each step touches; a successful run would prove only that the statements are valid for your schema, not that the new model is better.
-- documented shape; not run here
-- 1. new column, empty
ALTER TABLE guidance_passages ADD COLUMN embedding_v2 vector(384), ADD COLUMN embedding_model_v2 text;
-- 3. backfill one batch; the job repeats until no row is missing (vectors computed by the application, bound as $1, $2)
UPDATE guidance_passages SET embedding_v2 = $1, embedding_model_v2 = $2 WHERE passage_id = $3 AND embedding_v2 IS NULL;
-- 4. index the new column without blocking writes
CREATE INDEX CONCURRENTLY guidance_v2_hnsw ON guidance_passages USING hnsw (embedding_v2 vector_cosine_ops);
-- 6. after cutover, reads use the new column, guarded by its model
SELECT passage_id, embedding_v2 <=> $1 AS cosine_distance
FROM guidance_passages
WHERE status = 'current' AND embedding_model_v2 = $2
ORDER BY embedding_v2 <=> $1
LIMIT 50;
Step 1 is quick even on a large table: PostgreSQL's ALTER TABLE page (read on 2026-11-18) says adding a column with no default requires no rewrite of the table; existing rows read the new column as null. The new column may have a different dimension; the two are never compared. pgvector's README documents CREATE INDEX CONCURRENTLY for production and COPY for bulk loads, with indexes added after the initial data.
The lab: step through a migration#
The lab walks the same sequence with toy encoders, shows which column and query encoder readers use at each step, and has a "classic bug" checkbox for the Friday release. Toy encoders and synthetic data show how the mechanism works, not how good any model or product is.
The same sequence without JavaScript, with the numbers from the run in the next section (in this example):
| Step | Reads use | Query encoder | What a failure here costs |
|---|---|---|---|
Add column embedding_v2 | old column | toy-a@1 | nothing; nobody reads it |
| Dual write + backfill | old column | toy-a@1 | nothing; 18 rows backfilled, 1 dual-written, 19 in both |
| Index on new column | old column | toy-a@1 | build time only |
| Evaluate (14 labeled queries) | old column | toy-a@1 | nothing; old ndcg@5 0.678, new 0.553 |
| Cutover (only if the gate passes) | new column | toy-a@2 | rollback = switch back |
| Classic bug: new encoder, old column | old column | toy-a@2 | the guard refuses; without it, wrong neighbors |
| Drop old column | new column | toy-a@2 | rollback now needs a re-embedding |
The migration, run on the example#
The program below runs the seven steps on the example's 18 guidance passages, with toy-a@1 as the old model and toy-a@2 as the new version (it adds a query prefix and stops folding synonyms). A successful run proves that the sequence keeps both columns complete, that the guard refuses a mismatch and that rollback restores the original results; it says nothing about a real model, and the metrics come from synthetic labels.
# migrate.py, run from a scratch copy of examples/rag-uw
# An embedding migration in miniature: second column, dual write, backfill, evaluation gate,
# cutover, rollback. Toy encoders stand in for models; the labels are synthetic.
from data import GUIDANCE_QRELS, guidance_passages
from encoders import TOY_A1, TOY_A2, EmbeddingMismatch
from metrics import evaluate
from search import VectorTable, search_text
rows = guidance_passages()
old = VectorTable.build("guidance", rows, "passage_id", "text", TOY_A1) # column embedding (toy-a@1)
new = VectorTable("guidance", old.rows, "passage_id", "text", TOY_A2.name) # column embedding_v2, empty
reads = old # where queries go today
def write_passage(p):
"""Dual write: while the migration runs, every new or changed row gets BOTH vectors."""
old.add([p])
new.vectors[p["passage_id"]] = TOY_A2.encode_document(p["text"])
def backfill(batch=5):
"""Fill the new column for rows that do not have it yet, in batches (resumable)."""
done = 0
while missing := [r for r in old.rows if r["passage_id"] not in new.vectors][:batch]:
for r in missing:
new.vectors[r["passage_id"]] = TOY_A2.encode_document(r["text"])
done += len(missing)
return done
def score(table):
runs = {q["qid"]: [h["id"] for h in search_text(table, q["query"], 5).hits] for q in GUIDANCE_QRELS}
return evaluate(runs, {q["qid"]: q["rel"] for q in GUIDANCE_QRELS}, k=5)
before = [h["id"] for h in search_text(reads, "maximum roof age in wind zone C", 3).hits]
note = next(r for r in rows if r["passage_id"] == "GD-NOTE-08")
write_passage({**note, "passage_id": "GD-NOTE-09", "text": "Roof photos older than a year need a new inspection."})
print(f"1. dual write during backfill: GD-NOTE-09 has old={'GD-NOTE-09' in old.vectors} new={'GD-NOTE-09' in new.vectors}")
print(f"2. backfill: {backfill()} rows embedded with {TOY_A2.name}; rows in both columns: {len(old.vectors)} = {len(new.vectors)}")
try:
search_text(old, "maximum roof age in wind zone C", 3, encoder=TOY_A2)
except EmbeddingMismatch as e:
print("3. classic bug refused (new query encoder, old column):", str(e).split(":")[0])
for name, table in (("old", old), ("new", new)):
m = score(table)
print(f"4. gate on {len(GUIDANCE_QRELS)} labeled queries, {name} column ({table.embedding_model}): "
f"recall@5={m['recall@5']:.3f} mrr={m['mrr']:.3f} ndcg@5={m['ndcg@5']:.3f}")
reads = new if score(new)["ndcg@5"] >= score(old)["ndcg@5"] else old
print(f"5. cutover decision: reads use {reads.embedding_model}")
reads = new # rollback drill: cut over ...
reads = old # ... then point reads back; the old column was never touched
after = [h["id"] for h in search_text(reads, "maximum roof age in wind zone C", 3).hits]
print(f"6. rollback: same top 3 as before the migration: {after == before} {after}")
1. dual write during backfill: GD-NOTE-09 has old=True new=True
2. backfill: 18 rows embedded with toy-a@2; rows in both columns: 19 = 19
3. classic bug refused (new query encoder, old column): index built with toy-a@1, query encoded with toy-a@2
4. gate on 14 labeled queries, old column (toy-a@1): recall@5=0.774 mrr=0.738 ndcg@5=0.678
4. gate on 14 labeled queries, new column (toy-a@2): recall@5=0.637 mrr=0.681 ndcg@5=0.553
5. cutover decision: reads use toy-a@1
6. rollback: same top 3 as before the migration: True ['GD-NOTE-02', 'GD-2025.4-R-ROOF-01', 'GD-2026.2-R-ROOF-01']
Three results matter. The passage written during the backfill landed in both columns, so the new column was complete when the backfill ended. The guard refused the Friday release's combination before any distance was computed. And the gate kept reads on the old column, because the new version scored lower on the labeled queries: toy-a@2 no longer folds synonyms, so "aged roofing" stops landing near "older roof". A newer version is not automatically a better one, and only labels can say which. A simplified demonstration leaves things out: there is no concurrency, no crash halfway through the backfill and no index build, and a production gate also compares slices and latency, as on evaluation.
Common confusion. A migration re-embeds; it never reindexes its way to a new model. An index built over old vectors cannot be converted into an index over new ones, because the coordinates themselves change. Reindexing rebuilds a search structure from the same vectors; re-embedding recomputes the vectors. The scale page shows that growth needs neither.
Re-embed or reindex: two jobs, two runbooks#
Because the two are confused so often, each should have its own trigger and its own runbook. Re-embedding is needed only when the vectors would come out different: a new model or version, a new preprocessing step, a new template for the embedded text, or a new chunking rule. Reindexing is needed when the search structure no longer serves the same vectors well: recall measured against exact search has fallen after heavy churn, an index parameter is changed, or vacuum of a graph index takes too long. pgvector's README suggests REINDEX INDEX CONCURRENTLY before vacuuming an HNSW index for that last reason.
| Trigger | Re-embed? | Reindex? | In the example |
|---|---|---|---|
| Corpus grows 40 → 4,000 precedents | no | maybe, if measured | UW-R10: 3,960 new vectors, 40 unchanged |
| New model or model version | yes, all rows, as a migration | yes, on the new column | toy-a@1 → toy-a@2 above |
| New template for the embedded text | yes | yes | a new banded profile template for Corpus A |
| Index recall fell after many deletes | no | yes | measured against exact search |
hnsw.ef_search changed | no | no | a query setting, applied per query |
To know which job applies, record per row the model and version that produced the vector and the version of the text template or chunking rule. Without them, the only safe answer to "which rows are stale?" is "all of them".
Bulk updates and policy revisions as transactions#
Some changes arrive in bulk. A new policy version publishes every rule at once; a year of precedents is imported; a template change touches every profile. The operational question is what readers see while the change is half done.
When rows and vectors live in the same database, the answer can be "nothing": the change runs in one transaction, and readers see the old state until it commits. The example's SQL shows it with a policy revision. Query Q14 marks the current R-ROOF-01 passage as superseded inside a transaction, reads, and rolls back. It ran on a throwaway PostgreSQL 16.14 with pgvector 0.8.3 (one run on one laptop, toy data; not a benchmark). What it proves: inside the transaction both versions show as superseded, and after the rollback the current passage is back, so a reader outside the transaction never saw a state with no current roof rule.
-- ran: examples/rag-uw/sql/queries.sql, Q14 (output recorded in sql/OUTPUT.md)
BEGIN;
UPDATE guidance_passages SET status = 'superseded' WHERE policy_version = '2026.2' AND rule_id = 'R-ROOF-01';
SELECT passage_id, status FROM guidance_passages WHERE rule_id = 'R-ROOF-01' ORDER BY passage_id;
ROLLBACK;
SELECT passage_id, status FROM guidance_passages WHERE rule_id = 'R-ROOF-01' ORDER BY passage_id;
BEGIN
UPDATE 1
passage_id | status
---------------------+------------
GD-2025.4-R-ROOF-01 | superseded
GD-2026.2-R-ROOF-01 | superseded
ROLLBACK
passage_id | status
---------------------+------------
GD-2025.4-R-ROOF-01 | superseded
GD-2026.2-R-ROOF-01 | current
A real policy publish does the reverse in one transaction: insert the new version's passages with their vectors, mark the old version superseded, commit. Retrieval filters on status = 'current', so the switch is atomic for every reader: freshness by construction, and UW-R03's stale 30-year rule cannot appear in a filtered query.
When vectors live in a separate store, the same publish is two writes in two systems with a window between them; the storage page describes the outbox pattern that orders them.
Deleting a document everywhere#
A broker withdraws a submission and asks for its attachment to be removed. The document has several copies, and a delete that reaches only some of them leaves retrievable text behind. List them first: the source row; its chunks; one vector per chunk, per model during a migration; index entries; rows in a separate store; cached results and answers that quoted it; traces and evaluation sets that stored its text. Each copy needs its own delete, or a rule that makes it unreachable.
Diagram: Deleting a document everywhere: who removes each copy
Read left to right. Inside one database a foreign key with a cascade removes the chunks with their document, and a vector in the row goes with the row. A SQLite virtual table lives in the same file but takes no part in cascades, so the application deletes its rows explicitly. Copies outside the database learn about the delete from a tombstone. A reconciliation check proves that no orphan is left.
- One transaction
- Source document row: The system of record; its id leads to every copy.
- Chunks and in-row vectors: A foreign key with a cascade deletes the derived chunks in the same transaction.
- Same file, no cascade
- Virtual-table vectors: A vec0 table does not take part in foreign-key cascades.
- Outside the database
- Tombstone: A kept record of the delete that downstream jobs read.
- Separate stores and caches: A vector store, a cached answer and a stored trace do not know the delete happened.
- Proof
- Reconciliation check: No vector without a chunk, no chunk without a current document.
The diagram groups the copies by who removes them. Inside one database a foreign key with a cascade removes chunks with their document. Outside it, nothing happens unless a job acts. SQLite virtual tables are the trap in the middle: they live in the same file, yet they do not take part in foreign-key cascades.
The example's recorded sqlite-vec run (sqlite-vec 0.1.9, SQLite 3.53.4, throwaway environment) shows the trap: deleting a file cascaded to its chunk, but the chunk's vector stayed until an explicit delete.
Deleted one file: chunks cascade, vec0 rows do not. Orphaned vector rows: 1 of 6.
After an explicit DELETE on vec_chunk: 5 rows remain.
In Mnemos. In the code read, Mnemos keeps vectors in a separatevec0virtual table joined tochunkby id, and its delete paths clearvec_chunkexplicitly, because virtual tables have no foreign-key cascade.
For copies that a single transaction cannot reach, the delete leaves a record behind. A tombstone is a small row that says "document X was deleted at time T", kept after the document itself is gone. Downstream jobs read tombstones to delete their copies: the vector store, the cache, a replica. A tombstone also stops a lagging ingestion job from re-inserting a document that was deleted while it ran. Then a reconciliation check proves the result: no vector without a chunk, no chunk without a current document. The query below is a documented shape for the chunk and vector tables of the sqlite page; it was not run here. A successful run returns zero rows; any row is an orphaned vector that a query could still match.
-- documented shape; not run here
SELECT v.chunk_id FROM vec_chunk AS v LEFT JOIN chunk AS c ON c.id = v.chunk_id WHERE c.id IS NULL;
Production pitfall. An index can keep a deleted row's entry until maintenance runs: PostgreSQL's documentation says VACUUM removes dead row versions in tables and indexes. A separate engine with its own index needs its own delete call, and a cache in front of either has no idea a delete happened. Test deletes end to end: delete, then query with the deleted document's own text, and expect it nowhere.
Access filters inside the query, and tenant isolation#
The insurer writes business through more than one book, and the precedents of one book must not inform recommendations in another. This site's own case UW-R07 is a precedent, PRE-003, that belongs to book-west and leaks into a search for book-east.
The rule: an access filter must run inside the query, as a condition every candidate row must pass before ranking, never as a step that removes rows afterwards. An ACL filter (ACL stands for access control list) is that condition: it restricts a search to the rows the caller may see. Tenant isolation is the stronger property built from it: no query on behalf of one tenant can return, rank or count another tenant's rows.
Why "inside"? A filter applied after the top k does harm both ways. Other-tenant rows that ranked high took slots and are removed afterwards, so the result is short: post-filter starvation. And a code path that skips the post-filter returns them: a leak. The example shows both.
/opt/homebrew/bin/python3 run_demo.py --case UW-R07 --verbose
PASS UW-R07 Tenant filter inside the query
no tenant filter: returned other-tenant rows ['PRE-003']
tenant filter inside the query: only book-east rows (True)
post-filter after top 5: 2 rows (starved); pre-filter: 5 rows
SUMMARY: 1/1 cases PASS - all good
In SQL the filter is a bound parameter in the WHERE clause of the vector query, as in the example's Q5: WHERE pr.status = 'active' AND pr.tenant = $2. The tenant value must come from the authenticated session, never from text the model or the user wrote. With an approximate index the pgvector page shows why the filter can still starve the result, and which fixes keep it inside the search: iterative scans, an exact sort over the filtered rows, or partitions per tenant.
PostgreSQL can enforce the boundary a second time, below the application. Its row-level security, as documented on 2026-11-18, restricts which rows each role can read once ALTER TABLE ... ENABLE ROW LEVEL SECURITY is set; with no policy the default is deny; superusers and roles with BYPASSRLS bypass it, and table owners do too unless FORCE ROW LEVEL SECURITY is set. With an application role that does not bypass it, a forgotten WHERE tenant = $2 fails closed instead of leaking. It is defense in depth, not a substitute for the filter; access leaks as a guardrail are on guardrails.sammuthu.com.
The lab's operations preset holds four failures that operations owns: the mismatched encoder, the permission leak, the stale policy and too small a candidate set. Each shows the broken trace, the step at fault, the fix and a test that fails on the broken run and passes on the fixed one.
The same scenarios without JavaScript (in this example):
| Failure | Case | Step at fault | Test (fails broken, passes fixed) |
|---|---|---|---|
| Query and index from different encoders | UW-R06 | encode | The dense top 5 equals the exact top 5 with the index's own encoder |
| Another tenant's precedent leaks | UW-R07 | filter | Every returned precedent belongs to the querying tenant |
| Stale policy in the context | UW-R03 | filter | No superseded passage in the selected context |
| Inadequate recall: k too small | UW-004 | candidates | The recommendation is escalate and cites R-ROOF-03 |
Caches, and the semantic cache that answers the wrong question#
Retrieval systems cache query vectors, retrieved row ids and generated answers. The first is safe if its key includes the model identity; the other two copy data whose truth can change. A semantic cache goes further: it reuses a stored answer when a new query's vector is close enough to a cached query's vector. It saves a retrieval and a model call for paraphrases, and inherits every blind spot of the embedding. The program below demonstrates two failures with the toy encoder and then the key that prevents both. It proves the mechanism on toy vectors; the threshold of 0.05 is illustrative, and a real model's blind spots differ in detail.
# cache_check.py, run from a scratch copy of examples/rag-uw
# A semantic cache returns a stored answer when a new query's vector is close to a cached one.
# This shows two ways it serves a wrong answer, and the key that prevents both. Toy encoder, synthetic.
from encoders import TOY_A1
from vecmath import cosine_distance
THRESHOLD = 0.05 # "close enough" to reuse an answer (illustrative)
def lookup(cache, query, key):
q = TOY_A1.encode_query(query)
for (k, vec), answer in cache.items():
if k == key and cosine_distance(q, vec) <= THRESHOLD:
return answer, round(cosine_distance(q, vec), 4)
return None, None
def store(cache, query, key, answer):
cache[(key, tuple(TOY_A1.encode_query(query)))] = answer
naive = {} # key = embedding model only
store(naive, "maximum roof age in wind zone C", ("toy-a@1",), "30 years (R-ROOF-01, UW-GUIDE 2025.4)")
print("naive, after 2026.2 took effect:", lookup(naive, "max roof age allowed for wind zone C", ("toy-a@1",)))
print("naive, numbers are invisible: ",
lookup(naive, "is a roof older than 25 years allowed in wind zone C", ("toy-a@1",)))
keyed = {} # key = embedding model + policy_version + tenant
store(keyed, "maximum roof age in wind zone C", ("toy-a@1", "2025.4", "book-east"), "30 years (2025.4)")
print("keyed, current version 2026.2: ",
lookup(keyed, "max roof age allowed for wind zone C", ("toy-a@1", "2026.2", "book-east")))
print("keyed, other tenant: ",
lookup(keyed, "maximum roof age in wind zone C", ("toy-a@1", "2025.4", "book-west")))
naive, after 2026.2 took effect: ('30 years (R-ROOF-01, UW-GUIDE 2025.4)', 0.0023)
naive, numbers are invisible: ('30 years (R-ROOF-01, UW-GUIDE 2025.4)', 0.0374)
keyed, current version 2026.2: (None, None)
keyed, other tenant: (None, None)
The first naive lookup serves the superseded 30-year answer after the 25-year rule took effect: UW-R03 again, now from a cache instead of an index. The second is worse: a different question, about 25 years, receives the 30-year answer, because the toy encoder cannot see numbers (embeddings explains why real models see them poorly too). The keyed cache misses in both cases, which is correct. The rule: a cache key must include everything that changes the right answer, at least the embedding model, the policy version or a corpus generation number, and the tenant, and the cache must be invalidated when any of them changes.
Observability: log the retrieval, not only the answer#
When an underwriter disputes a recommendation, the first question is what the model read. A log of the final answer cannot say whether the wrong passage came from a stale filter, a mismatched encoder, a short candidate list or the model. The retrieval trace, defined on pipelines, records one entry per stage and answers it. An excerpt for SUB-1004's guidance search, from pipeline.py --submission SUB-1004 --verbose run in a scratch copy (trimmed to three stages):
{"stage": "encode", "model": "toy-a@1", "index_model": "toy-a@1", "dims": 16, "guard": "checked"},
{"stage": "filter", "filter": "policy_version = current OR attachment of this submission",
"where": "pre", "rows_total": 18, "rows_after": 13},
...
{"stage": "select", "context_n": 5, "threshold": null,
"selected": ["GD-2026.2-R-ROOF-01", "GD-2026.2-R-ROOF-03", "GD-2026.2-R-ROOF-02", "GD-NOTE-01", "GD-NOTE-02"]}
Each field answers one operational question. model and index_model show the migration state and catch UW-R06. where: pre and the row counts show the filter ran inside the query and how selective it was. selected names exactly what the model read. Store the trace with the run's run_id and trace_id, keep the tenant and policy version on it, and alert on the signals that precede failures: guard refusals, short results, and a rising share of queries that return fewer than k rows. Traces as a discipline, with spans, sampling and retention, belong to observe.sammuthu.com.
In a coding agent. A coding agent's repository index has the same day-two problems. After a refactor, chunks of deleted files must lose their vectors, or the agent cites code that no longer exists; a cache of "where is the retry logic" answers must be keyed by the commit it was computed at; and switching the embedding model means re-embedding every chunk, behind a guard, before any query uses the new encoder. Logging which chunks each answer read is what makes a wrong edit explainable.
What costs money, in words#
No prices appear on this site; the drivers are stable and can be reasoned about. Embedding compute grows with the number of texts embedded: the initial load, every new or changed row, and every migration, which re-embeds everything once. Storage grows with rows times dimension times bytes per value, and doubles for the length of a migration, while two columns exist. Index memory and build time grow with rows and with graph settings, as the scale page computes. Query compute grows with queries per second times the work per query: the first stage, the reranker over the candidate pool, and the model call over the context. Operations has its own costs: backfill jobs, reconciliation jobs, trace storage and the people who read them.
Two habits keep these drivers in check. Never re-embed by accident: growth and reindexing do not need it. And embed each distinct text once; the insurance build embeds 737 distinct texts for its 1,140 profile rows.
What products document about these operations#
Managed products document their update models differently, and the runbook must follow them. Google's Vector Search, as documented on 2026-11-18, offers batch updates or streaming upserts, fixes that choice when the index is created, and lets restricts (its filter tokens) be updated without re-embedding; see Google Cloud. For the products on ecosystems, find the answer to "when is a write searchable" in the documentation, or measure it.
A checklist for Monday#
| Question | Why it matters |
|---|---|
Is embedding_model stored per vector and checked on every query? | The guard turns UW-R06 into a refusal instead of wrong neighbors |
| Does the runbook move the query encoder and the column in one switch? | The Friday release moved one without the other |
| Is the old column kept, and dual-written, until the rollback window ends? | Rollback should be a switch, not a second migration |
| Does a labeled evaluation gate the cutover? | A newer model version can score lower on your queries |
| Does a delete reach chunks, vectors, indexes, stores and caches, with a reconciliation check? | Virtual tables and separate stores do not cascade |
| Do tenant and access filters run inside the query, from the session? | Post-filters leak or starve (UW-R07) |
| Do cache keys include model, policy version and tenant? | A semantic cache serves superseded or foreign answers |
| Is the retrieval trace stored with the run? | It is the only record of what the model read |
Two columns or two tables for a migrationIn-place columns, a shadow table, or a second index behind an alias
The column design on this page is the simplest when vectors live in the source row: one table, one transaction, two vectors. It has costs. Every row grows by a second vector for the whole migration. Filling the column is an UPDATE per row, and PostgreSQL's documentation says an update does not immediately remove the old row version; VACUUM must reclaim it. Dropping the old column later is quick, but the same documentation says the space is not reclaimed until rows are rewritten.
A shadow table keyed by row id and model avoids touching the source table at all. The backfill inserts into it, the index is built on it, and the cutover changes the join. Rollback is changing the join back, and the drop is DROP TABLE, which removes the old vectors without leaving dead space in the source table. The auxiliary layout from the storage page makes this the natural design.
A separate vector engine usually offers a second collection or index plus some way to point readers at one or the other; whether it has an alias or similar switch is a product question to check in its documentation. Whatever the mechanism, the invariant is the same: two complete sets of vectors, one switch that moves the query encoder and the target together, and a rollback that moves both back.
Backfills that can crash, overlap and throttleSelect by what is missing, write idempotently, protect the live system
A backfill of a million rows runs for hours, competes with live traffic and will crash at least once. Three properties make it safe to restart at any point. It selects work by state, "rows whose new vector is missing or older than the source text", never by offset, so a restart picks up exactly what is left. Each write is conditional, WHERE embedding_v2 IS NULL or a version check, so two overlapping workers cannot overwrite a newer vector with an older one. And it records progress outside the job, so operators can see how many rows remain.
The race to think about is the backfill against dual write. A row is changed while the backfill is computing its new vector from the old text. If the backfill writes last, the new column holds a vector of stale text. The fix is a version or content hash on the source row: the backfill writes only if the hash it embedded still matches. The insurance build's embedding_version per row is a field of exactly this kind.
Throttling protects the live system: limit batch size, pause on replication lag or rising p95 latency, and respect the embedding provider's rate limits if the model is remote. A local model in process, as in the insurance build, removes the remote rate limit but not the CPU contention.
Tenant isolation: filter, partition, or separate databaseThree designs and what each one fails at
A shared table with a tenant column and a filter in every query is the cheapest design and the easiest to get wrong: one query path without the filter leaks. Row-level security in PostgreSQL makes that path fail closed, provided the application role does not bypass it. With an approximate index, a selective tenant filter also starves results, so the filter design and the search design must be chosen together.
A partition per tenant makes the boundary structural. The planner skips other tenants' partitions before any distance is computed, each partition has its own index, and dropping a tenant is dropping a partition. It costs more objects to manage, and very many small tenants make it awkward.
A database or engine per tenant gives the strongest isolation and the clearest deletion story: delete the database. It multiplies everything else: migrations run per tenant, indexes are built per tenant, and cross-tenant analytics become a separate system. The choice is usually made by compliance and contract, not by search performance. The ecosystems page compares how products express tenancy; the two-corpora page weighs one cluster against several.
Invalidating caches by generation, not by guessA corpus generation number makes every cached answer checkable
Time-based expiry is the wrong tool for a corpus whose truth changes at known moments. A policy version takes effect on a date; a precedent is revoked at a time. A cache entry that expires after an hour serves the old rule for up to an hour after the change, and a shorter expiry only shrinks the window.
A generation number closes it. Keep one counter per corpus and tenant, increment it in the same transaction as every change that could alter retrieval: a publish, a revoke, a delete, a cutover. Put the counter in every cache key. The moment the change commits, every older entry becomes unreachable, with no scan and no guess. The cost is a lower hit rate right after each change, which is the correct trade for a system whose answers must follow current policy.
Answers that cite evidence add one more check: before serving a cached answer, confirm that every cited passage or precedent is still current and visible to the caller. That check reads the system of record, and it is cheap compared with serving a revoked precedent to an underwriter.
In one breath#
A retrieval system stays correct only if every change reaches every copy of the data: a model change runs as a migration with two complete sets of vectors, a guard, a labeled gate and a rollback that is a switch; a delete reaches chunks, vectors, indexes, stores and caches, and a reconciliation check proves it; access and tenant filters run inside the query from the session. Policy revisions are safest as transactions in the database that holds the vectors, and caches must be keyed by everything that changes the right answer. The retrieval trace is the record that makes all of this auditable.
The insurer's corpora can now change underneath the agent without returning stale, foreign or mismatched rows, and a disputed recommendation can be traced to the rows it read. What remains open is the quality of a single retrieval when the query itself is the problem: a vague question, a rule split across sections, a question about what was in force last year. The next page, advanced retrieval, takes the techniques built for those cases one at a time, with what each costs.