The insurer holds an architecture review before the underwriting agent's retrieval goes to production. Three teams present. One draws a box labeled "vector DB" with arrows to the agent. One draws PostgreSQL with three tables. One draws four Google Cloud products. The reviewer asks each team the same two questions: why is each box there, and how do you know it returns the right rows?
None of the three can answer the second question with a number. The first team has never compared its results with an exact search. The second has an HNSW index but has not checked whether the planner uses it. The third does not know whether its filters run before or after the approximate search, which for the tenant filter of this site's case UW-R07 decides whether another book's precedents can leak.
A diagram of boxes is not an architecture until each box has a reason and each claim of quality has a measurement. This page presents four designs in that form. Each has a diagram, the decisions with their reasons, what to measure, and a benchmark recipe built from the example, so a team can rerun it on its own data. A reference architecture is that package: a whole design with its decisions, its trade-offs and the way to verify it, written so another team can adapt it rather than copy it.
What this is not. How retrieval sits inside the agent runtime, with its tool loop and budgets, is harness.sammuthu.com/rag/. Product capabilities in full are on Google Cloud and ecosystems; this page uses them, it does not re-derive them.
The decisions every design makes#
Retrieval is a query. The model reads only what it returns. Every design on this page implements that sentence: an embedding column holds one vector per row, the query vector is compared with the rows' vectors, the rows are ranked by distance and the nearest k are returned. The designs differ in five decisions, and the first three are independent of each other, which is the separation the whole site defends.
The storage layout says where the vector lives: in the source row, in an auxiliary table joined by row id, in a virtual table, or in a separate engine. The search algorithm says how rows are compared: exact, every candidate scored, or approximate through an index such as HNSW. The embedding model says how coordinates are made, and with which dimension and metric. Two more decisions follow from the use case: where the filters run (inside the search, or after it), and where the data lives, which is data residency: the machine, region or jurisdiction where text, vectors and every copy are stored and processed.
The lab shows the first decision for the same query under each design. Pick an architecture and it selects the layout: Mnemos uses an auxiliary vector table, the insurance build a vector column in the row, the managed stack a managed store, the Postgres-native service a column in the row or an auxiliary table. The row id of the chosen match is highlighted from the match to its source row. Toy encoders and synthetic data show how the mechanism works, not how good any model or product is.
The same matches without JavaScript (query = SUB-1004's banded profile, toy-a@1, exact search, filter wind_zone = 'C', in this example):
| Match | Cosine distance | Source row | Vector column in the row | Auxiliary vector table | Managed store |
|---|---|---|---|---|---|
PRF-0218 | 0.0018 | SUB-2218, office, masonry, built 1994 | the match is the row | join on profile_id | fetch by id |
PRF-0206 | 0.0045 | SUB-2206, retail, frame, built 1960 | same | same | same |
PRF-0091 | 0.0048 | SUB-2091, office, concrete, built 1979 | same | same | same |
The three matches are the same in every column: the layout changes how a match leads back to its source row and what an update or a delete must touch, not which rows are nearest.
One benchmark recipe for all four designs#
Before the designs, the yardstick. Each design must answer the reviewer's second question the same way, so the recipe is shared. It follows step 1 to 3 of the scaling playbook on scale, and it measures three things the example can compute honestly: index recall (of the exact top k, the share a plan returned), recall@k against human labels, and distance computations as the unit of work. It never reports milliseconds from toy code.
- Hold out queries. Use real queries, or rows held back from the index, never queries that are themselves in the index.
- Compute the answer key with exact search, with the same filters the production query carries.
- Measure each candidate plan against it: index recall@k, distance computations per query, and short results (queries that returned fewer than k rows).
- Measure recall@k against labels for each slice that matters, as on evaluation: current-version guidance, one tenant, one wind zone.
- Record the configuration (model, metric, index settings, filter, data version) next to every number, and rerun after every change.
The program below applies steps 1 to 3 to Corpus A of the example: 20 profiles held out as queries, 220 stored, a toy HNSW and the wind_zone = 'C' filter of UW-R04. It demonstrates the recipe's shape and shows how a filter changes the answer. A successful run proves only how these plans behave on 220 banded toy vectors; it is not a benchmark of any engine, and many banded summaries are identical, which makes recall easy here.
# arch_bench.py, run from a scratch copy of examples/rag-uw
# One benchmark recipe for any of the four architectures, run here on Corpus A of the example:
# hold out queries, compute the exact answer key, then measure each candidate plan by index recall@k,
# distance computations (work, not time) and short results. Toy encoder, synthetic rows.
from ann import HNSW, index_recall
from data import applicant_profiles
from encoders import TOY_A1
from vecmath import cosine_distance
K = 5
rows = applicant_profiles()
held_out, stored = rows[:20], rows[20:] # queries are never inserted into the index
points = [TOY_A1.encode_document(r["summary"]) for r in stored]
queries = [TOY_A1.encode_query(r["summary"]) for r in held_out]
zone_c = lambda i: stored[i]["wind_zone"] == "C" # the filter UW-R04 carries
print(f"stored rows {len(stored)}, held-out queries {len(queries)}, rows passing wind_zone = 'C': "
f"{sum(zone_c(i) for i in range(len(stored)))}")
def exact(q, flt):
"""The answer key: compare with every row that passes the filter."""
cand = [i for i in range(len(points)) if flt is None or flt(i)]
ranked = sorted((cosine_distance(q, points[i]), i) for i in cand)[:K]
return [i for _, i in ranked], len(cand), {}
index = HNSW(points, M=8, ef_construction=32, distance=cosine_distance)
plans = [("exact scan (answer key)", lambda q: exact(q, None), None),
("HNSW ef_search=10", lambda q: index.search(q, K, 10), None),
("HNSW ef_search=40", lambda q: index.search(q, K, 40), None),
("zone C: HNSW, post-filter, ef=10", lambda q: index.search(q, K, 10, flt=zone_c), zone_c),
("zone C: HNSW, iterative, ef=10", lambda q: index.search(q, K, 10, flt=zone_c, iterative=True), zone_c),
("zone C: exact over filtered rows", lambda q: exact(q, zone_c), zone_c)]
print(f"{'plan':<34} {'recall@5':>8} {'work/query':>10} {'short':>5}")
for name, search, flt in plans:
r = index_recall(search, points, queries, K, cosine_distance, flt=flt)
print(f"{name:<34} {r['recall']:>8.2f} {r['distance_computations']:>10.1f} {r['short_results']:>5}")
print(f"index build work: {index.build_computations} distance computations")
stored rows 220, held-out queries 20, rows passing wind_zone = 'C': 104
plan recall@5 work/query short
exact scan (answer key) 1.00 220.0 0
HNSW ef_search=10 1.00 54.9 0
HNSW ef_search=40 1.00 101.3 0
zone C: HNSW, post-filter, ef=10 0.83 54.9 10
zone C: HNSW, iterative, ef=10 1.00 82.8 0
zone C: exact over filtered rows 1.00 104.0 0
index build work: 55561 distance computations
Read the rows as a reviewer would. Unfiltered, the toy index matched the answer key with a quarter of the work. With the zone filter applied after the walk, half the queries came back short and recall dropped to 0.83, the starvation of UW-R09. An iterative scan recovered full recall at more work than the unfiltered walk, and exact search over the 104 filtered rows reached full recall with no index at all. Which plan a design should use is decided by that table on its own data, and it is the table each design below must be able to produce.
Design 1: Mnemos, local-first on one file#
The situation. A person wants to ask questions of private notes and documents without sending them to a service. Mnemos is an open-source local-first personal RAG application by this site's author (TypeScript, Next.js); the facts below are from its code, version 0.21, as read. Local-first means the data and the computation stay on the person's machine by default.
Diagram: Design 1, Mnemos: everything that touches private text runs on one machine
Read left to right. Files are chunked and embedded in process, chunks and vectors live in one SQLite file, and the query is an exact search joined back to chunk, file and source. The only optional path off the machine carries retrieved chunks to an external model, and the audit log records each query. Facts as in the code read (v0.21).
- On the machine
- Chunk and embed locally: Recursive character splitter; bge-small-en-v1.5 in process, mean pooling, normalized.
- One SQLite file: Vectors in a vec0 virtual table joined to chunk by id: an auxiliary vector table.
- Exact search: Default L2 on normalized vectors ranks like cosine; dense only, no reranker.
- Optional, logged
- External model, if configured: The corpus never leaves; only what retrieval returned, and every query is in the audit log.
The decisions, as in the code read. Storage is one SQLite file, opened through better-sqlite3 with the sqlite-vec extension loaded, with tables source, file and chunk. Vectors live in a separate virtual table, vec_chunk USING vec0(chunk_id INTEGER PRIMARY KEY, embedding FLOAT[384]), joined to chunk by id: an auxiliary vector table. The query is WHERE v.embedding MATCH ? AND k = ? ORDER BY v.distance, joined to chunk, file and source. Embeddings come from Xenova/bge-small-en-v1.5 (384 dimensions) through @xenova/transformers, run in process, with mean pooling and normalization. No distance_metric option is set on vec0, so it uses the documented default, L2; the code comments call the distance "cosine", and because vectors are normalized at embedding time, L2 and cosine rank the same. Chunking is a recursive character splitter of about 1,000 characters with about 200 overlap. Retrieval is dense only in the code read: no BM25, no reranker.
The reasons. One file is the whole system: backup is a copy, deletion is a delete, and nothing needs a server. Local embeddings and local chat are the default, and if an external model is configured it receives retrieved chunks only, while an audit log records each query. That is the data-residency story in one sentence: the corpus never leaves; only what retrieval returned, and only when configured, does.
What to measure. The recipe's step 3 is short here: exact search is the plan, so index recall is 1.0 by construction, and the work per query is the number of chunks. Measure recall@k on a few dozen of the person's own labeled questions, because dense-only retrieval misses exact names and identifiers (the reason hybrid exists), and check that deletes leave no orphaned vectors, since virtual tables have no foreign-key cascade; Mnemos's delete paths clear vec_chunk explicitly. The example's standard-library version on the sqlite page runs the same exact search with bound parameters and prints SQLite's plan, a full scan and a sort; the recorded sqlite-vec run confirms that the default vec0 distance and cosine distance give the same order on normalized vectors ("same order with both metrics: True").
In Mnemos. The design choice worth copying is the boundary, not the stack: everything that touches private text runs in process, and the one optional path out of the machine carries retrieved chunks only and is logged.
Design 2: the insurance dual-corpus design#
The situation. An underwriting agent needs two kinds of similarity: past applicants that look like this one (Corpus A) and how underwriters decided similar cases (Corpus B), plus the current policy text. The series example on this site reconstructs that workflow for commercial property; the insurance build is a separate, production-shaped Java service in its own domain, with simulated vendors. The series example is an illustrative reconstruction for teaching: cases, rules, fields, vectors and every number are invented, and nothing here reports the quality of a real system or model.
Diagram: Design 2, the insurance build: two corpora in one PostgreSQL, two search plans
Read top lane then bottom lane. Corpus A holds banded applicant profiles with an HNSW index and a btree filter; Corpus B holds decided precedents with no approximate index, an exact sort over a small filtered set and a feature reranker that returns both sides. Both use one in-process embedding model and cosine distance, and both join back to source rows. Facts as stated for that build.
- Corpus A: similar applicants
- Banded profile template: 1,140 rows, 737 distinct texts, versioned by embedding_version.
- Filter, then HNSW or exact: Btree on (state, zip3, dwelling_type); the planner chose the btree and an exact sort with the selective filter.
- Corpus B: decided precedents
- Exact sort, no ANN index: Btree on (provision_key, status, decided_at DESC), then distance order, LIMIT 50.
- Feature rerank, both sides: Drop below 4 of 8 bands; top 3 approve-side and top 3 decline-side.
- Shared
- all-MiniLM-L6-v2 in process: L2-normalized in code; cosine distance with vector_cosine_ops.
The decisions, in that build. It runs on PostgreSQL 16 with pgvector (the pgvector/pgvector:pg16 image), with vectors in the source rows. The embedding model is all-MiniLM-L6-v2 at 384 dimensions, run in process through ONNX Runtime, with no embedding API call; vectors are L2-normalized in code and compared with cosine distance <=> and vector_cosine_ops. Corpus A, customer_profiles, holds 1,140 seeded rows whose embedded text is a fixed banded template, never raw numbers or personal data, versioned by embedding_version; it has an HNSW index (m = 16, ef_construction = 64, hnsw.ef_search = 80, iterative scan relaxed_order when pgvector is 0.8 or later, then a re-sort) and a btree on (state, zip3, dwelling_type). Its query filters first, then orders by distance with LIMIT 8, and widens the filter step by step, reporting each step, when fewer than 3 rows come back. Corpus B, precedents, has 41 seeded rows marked simulated plus rows learned from human decisions, and no ANN index: a btree on (provision_key, status, decided_at DESC) and an exact distance sort over 50 candidates, then a feature reranker (0.6 × matching bands + 0.3 × cosine similarity + 0.1 × recency decay), dropping rows below 4 of 8 bands and returning the top 3 approve-side and top 3 decline-side, so the model sees both.
The reasons. Every corpus needs exact filters next to the vector ranking, precedents must be searchable the day they are decided, and every recommendation must join back to source rows for audit: one relational database gives all three in one transaction. Corpus B's sets per provision are small, so exact search is both correct and cheap; the schema notes HNSW as worth considering only past about 50,000 rows per provision. The reranker compares bands exactly because a banded profile through an embedding is a coarse signal; the example shows the same effect when a retail frame building ranks second for an office in UW-R04.
What to measure. That build's documentation reports what the planner chose: with the selective filter, the btree and an exact sort over about 190 rows; unfiltered at 1,140 rows, a sequential scan; the HNSW walk only with enable_seqscan = off. So the first measurement is the plan, then index recall of Corpus A's HNSW against exact search under each filter, then the golden sets: that build stores them in JSON, measures hit@1, hit@3, MRR and filter precision, stores each run, and fails the build on a regression. The recipe above is the toy version of the Corpus A part.
The lab puts the two corpora of the series example side by side for one submission, with metadata, permissions, policy_version and evidence ids.
The same retrievals without JavaScript (in this example):
| Submission | Corpus A top 5 (82 of 240 pass the filter) | Corpus B approve-side | Corpus B decline-side | Recommendation |
|---|---|---|---|---|
| SUB-1004 | PRF-0218, PRF-0206, PRF-0091, PRF-0172, PRF-0197 | PRE-001, PRE-023, PRE-030 (all 2025.4, flagged) | PRE-002 (override), PRE-005, PRE-007 | escalate, cites R-ROOF-03, EV-ROOF-0412 |
| SUB-1005 | PRF-0050, PRF-0007, PRF-0138, PRF-0047, PRF-0056 | PRE-004, PRE-030, PRE-023 (2025.4, flagged) | PRE-007, PRE-012, PRE-002 | decline, cites R-ROOF-01, EV-ROOF-0515; attachment flagged as untrusted |
| SUB-1007 | PRF-0050, PRF-0138, PRF-0007, PRF-0212, PRF-0047 | PRE-004, PRE-030, PRE-023 (2025.4, flagged) | PRE-007, PRE-002, PRE-012 | decline, cites R-ROOF-01 |
Every recommendation has requires_human: true. The model recommends; a person decides. The corpus contracts and the split-or-shared decision matrix live on two corpora.
Design 3: a managed Google Cloud stack#
The situation. The insurer's platform team runs on Google Cloud and prefers that the provider operate databases and indexes. A managed service is a product whose provider runs the servers, scaling and patching, and often the index or the pipeline; you call it through SQL or an API. Every product statement below is from the Google Cloud page and its fact log, as documented on 2026-11-18.
Diagram: Design 3, managed Google Cloud: corpora as tables, managed parts around them
Read left to right. The embeddings API turns text into vectors with task types; the three corpora stay tables in Cloud SQL or AlloyDB, queried with SQL filters and joins; the ranking API can rerank the candidates. Vector Search is the alternative for a corpus moved out of the database; its filter timing must be tested. Products as documented on 2026-11-18; no prices, no ranking.
- Managed model endpoints
- Text embeddings API: Corpus as RETRIEVAL_DOCUMENT, queries as RETRIEVAL_QUERY; set autoTruncate to false.
- Ranking API: Returns a 0-to-1 relevance score per record: an ordering, not a probability.
- Managed database
- Cloud SQL or AlloyDB: SQL filters inside the query, same-day writes, joins back to source rows; private IP via private services access.
- Alternative to test
- Vector Search index: Filter timing is not stated in the docs read; a brute-force index gives an answer key inside the product.
The decisions. The three corpora stay tables in one managed PostgreSQL database, Cloud SQL for PostgreSQL with pgvector (0.8.5 on PostgreSQL 13 and later) or AlloyDB for PostgreSQL, because every corpus needs exact filters, precedents need same-day writes, and audit needs joins back to source rows; this is the design the Google Cloud page reaches for these conditions, not a recommendation for every team. Vectors come from the text embeddings API, with the corpus embedded under the RETRIEVAL_DOCUMENT task type and queries under RETRIEVAL_QUERY, and autoTruncate set to false, because its default is true and a truncated passage fails silently. Reranking can use the ranking API, which reranks up to 1,000 records per request and returns a 0-to-1 relevance score per record. Connectivity uses private IP through private services access for both databases. Backups are the database's own: automated backups and point-in-time recovery on Cloud SQL, continuous backup on by default on AlloyDB.
The alternatives in the same stack. Vector Search can hold an index for a corpus the team decides to move out of the database, or one that needs its streaming updates; its filters are token and numeric restricts, and its filtering page does not say whether they apply during the approximate search or after, so the tenant restrict must be tested with UW-R07 before Corpus B goes there. RAG Engine runs ingestion, chunking, embedding and retrieval for you, but the pages read do not mention metadata filtering and list data residency as not supported, which keeps it away from tenant-scoped precedents until tested.
What to measure. The recipe applies to each product. In Cloud SQL or AlloyDB it is the SQL of design 4. In Vector Search the answer key can live inside the product: an index can be configured with bruteForceConfig instead of treeAhConfig, so the same queries against a brute-force index and an approximate one give index recall without leaving the service. Then measure what the documentation leaves open: filter timing under a selective restrict, and how long a streaming upsert takes to appear in results, which the documentation describes only as a short delay. Treat the ranking API's 0-to-1 score as an ordering, not a probability, as on reranking.
Design 4: a Postgres-native service at mid scale#
The situation. A team runs a retrieval service for several applications and tenants, on PostgreSQL it already operates. "Mid scale" here means more rows and traffic than one laptop, and not a size threshold: no row count decides this design.
Diagram: Design 4, Postgres-native: one engine, plans chosen by measurement
Read left to right. One table per corpus with a vector column, partitions for large tenants, exact search where filters are selective and HNSW where the recipe shows it is needed, full text and vectors fused with reciprocal rank fusion in SQL, and replicas for reads. The recipe runs in SQL against the database itself.
- Storage
- One table per corpus: Rows, vectors and metadata in one transaction; model identity stored per row.
- Partitions per large tenant: The planner skips other tenants before any distance is computed.
- Query
- Exact or HNSW, by EXPLAIN: Selective filters favor an exact sort; an index only where measured.
- Full text + vectors: Two ranked CTEs combined with reciprocal rank fusion.
- Measure
- Index recall in SQL: Index scans off for the answer key, then the HNSW query per ef_search.
The decisions. One table per corpus with an in-row vector column, or an auxiliary table keyed by row id and model where migrations are frequent. Exact search wherever filters are selective, with a btree on the filter columns; an HNSW index only where the recipe shows exact search misses the latency target. Tenancy through a tenant column in every query and, for large tenants, partitions; pgvector's README suggests partial indexes for a few filter values and partitioning for many. Hybrid search in SQL: a full-text ranking and a vector ranking fused with reciprocal rank fusion, as in the example's Q7. Reads that outgrow one server go to replicas, which pgvector's README lists for horizontal scaling. Model changes run as the migration on operations.
The reasons. One engine for rows, vectors, filters, full text and transactions keeps every query auditable with EXPLAIN, and every change atomic. The team's existing skills for backups, replicas and monitoring carry over.
What to measure. The recipe runs as SQL against the database itself. The example's Q11 ran on a throwaway PostgreSQL 16.14 with pgvector 0.8.3: it forces an exact top 10 with index scans off, then the HNSW top 10 at three ef_search values, and counts the overlap. A successful run gives index recall for one query on one table; a real recipe loops over held-out queries and slices. One run on one laptop, toy data; not a benchmark.
-- ran: examples/rag-uw/sql/queries.sql, Q11 (excerpt; output recorded in sql/OUTPUT.md)
SET enable_indexscan = off;
CREATE TEMP TABLE exact_top AS
SELECT id FROM scale_points ORDER BY embedding <=> :'sq'::vector(16) LIMIT 10;
RESET enable_indexscan;
SET enable_seqscan = off;
SET hnsw.ef_search = 10;
CREATE TEMP TABLE ann_10 AS SELECT id FROM scale_points ORDER BY embedding <=> :'sq'::vector(16) LIMIT 10;
-- ... the same for ef_search 40 and 100 ...
SELECT 10 AS ef_search, count(*) / 10.0 AS index_recall_at_10 FROM ann_10 JOIN exact_top USING (id)
UNION ALL SELECT 40, count(*) / 10.0 FROM ann_40 JOIN exact_top USING (id)
UNION ALL SELECT 100, count(*) / 10.0 FROM ann_100 JOIN exact_top USING (id);
ef_search | index_recall_at_10
-----------+------------------------
10 | 1.00000000000000000000
40 | 1.00000000000000000000
100 | 1.00000000000000000000
On that 20,000-row toy table every setting matched the answer key, which says the recipe works, not that recall will be 1.0 on real vectors. The same run's Q12 and Q13, shown on pgvector, complete the picture for a selective filter: 0 rows with iterative scan off, 10 rows with relaxed_order, and with a btree on the filter column the planner chose a bitmap scan of 200 rows and an exact sort.
Production pitfall. A design document that names an engine and an index but no measured plan is a guess with a diagram. In both PostgreSQL designs above, the planner may prefer a btree and an exact sort over the HNSW index, and that can be the right plan. Read EXPLAIN for each production query shape before claiming the index is used.
The four designs side by side#
With each design explained, the table compares them on the five decisions and on the first measurement each one owes the reviewer.
| Decision | Mnemos | Insurance build | Managed Google Cloud | Postgres-native |
|---|---|---|---|---|
| Storage layout | auxiliary vec0 table | vector column in the row | column in Cloud SQL or AlloyDB; optional managed index | column in the row or auxiliary table |
| Search | exact | Corpus A HNSW, Corpus B exact | exact or index, by measurement | exact where selective, HNSW where measured |
| Model | bge-small-en-v1.5, 384, in process | all-MiniLM-L6-v2, 384, in process | embeddings API with task types | team's choice, versioned per row |
| Filters | joined source rows | inside the SQL query | inside the SQL query; restricts to test | inside the SQL query; partitions per tenant |
| Data lives | one file, one machine | one PostgreSQL database | the chosen region | the team's database servers |
| First measurement | recall@k on personal labels | the planner's plan, then index recall | filter timing and freshness | index recall per slice in SQL |
Two conclusions follow. Three of the four designs keep the corpora as tables in a relational database, for the same reasons: exact filters, same-day writes and joins back to source rows; the fourth keeps them in a local SQLite file for a reason of its own, residency. And every design's first measurement is different, because each one hides a different uncertainty.
Common confusion. "We use pgvector" and "we use HNSW" describe two different decisions. The insurance build uses pgvector for both corpora and HNSW for only one of them; Mnemos uses an extension whose vec0 table this site treats as exact unless verified. Layout, algorithm and model are chosen separately, and a design description should name all three.
In a coding agent. A coding agent's repository index fits design 1 or design 4. Run locally, it is one file per repository with chunk vectors keyed by path, commit and line offsets, searched exactly; run as a team service, it is a PostgreSQL table with a repo filter inside every query and grep beside it for identifiers. The recipe is the same: held-out questions with known answer locations, an exact answer key, and recall@k per repository.
A checklist for Monday#
| Question | Why it matters |
|---|---|
| Does the design name its storage layout, search algorithm and model separately? | They are independent decisions with independent reasons |
| Is there an exact answer key for the production query shapes, with filters? | Index recall needs a ground truth |
Has EXPLAIN been read for each query shape? | The planner may not use the index, and that may be right |
| Do tenant and version filters run inside the search, and is that documented or tested? | A filter after top k leaks or starves |
| Where does every copy of the data live, including caches and traces? | Residency covers derived data |
| Is each decision written with its reason and its measurement? | Otherwise it is a box, not an architecture |
Mixing designs: relational corpora, managed partsWhich parts move to a service first, and what moves with them
Real systems rarely stay pure. A Postgres-native design may call a managed embeddings API, a managed reranker, or a managed search application for a separate human-facing search over policy documents, as the Google Cloud page suggests. Each managed part brings its own boundary: what it receives, where it runs, and what it hides.
The parts that move most easily are stateless: an embeddings endpoint and a reranker take text and return numbers, and can be swapped behind an interface, although a swap of the embedding model is still a full migration. The parts that move least easily hold the corpus and its filters, because they carry permissions and freshness. A rule that holds in all four designs: keep the system of record for tenant, status and policy version in one place, and make every other component re-check against it after it returns ids.
Moving between designs laterWhat is portable, and what must be recomputed
Teams often start with design 1 or 4 and later consider another. Text, metadata and row ids move with ordinary exports. Vectors move only if the destination uses the same model, version, preprocessing and metric; the stored embedding_model value is what proves it, and the guard of UW-R06 should refuse anything else. Indexes never move: the destination builds its own from the vectors.
Two details catch teams. Metrics differ by default: sqlite-vec's vec0 defaults to L2 and pgvector's operator decides the metric per query, so normalized vectors are the portable choice, because then L2, cosine and inner product rank the same. And filters change meaning: a SQL WHERE on a joined column is not the same as a restrict token in a managed index, so the tenant test must be rerun on the destination before any traffic moves.
Failure domains: what breaks, and what still answersRetrieval outages, partial results and the agent's behavior
Each design fails differently. A local-first file can be corrupted or lost with its disk, so backup is part of the design, not an afterthought. A single PostgreSQL primary takes the corpora down with it; replicas keep reads alive, and the agent should know whether it is reading a lagging replica when freshness matters. A managed design adds network paths and quotas, and a dependency on regions and allowlists.
What matters most is what the agent does when retrieval fails or returns too little. Short results under a filter, a timeout from a reranker, or an empty guidance search must lead to abstention or escalation, never to a recommendation built from parametric memory. The grounding page treats abstention; in every design the retrieval trace should show that the failure happened, so a person can see why the agent declined to recommend.
Traps in the benchmark recipeEasy data, leaked queries and one-number summaries
The run on this page reaches recall 1.0 for most plans, and that is a warning, not a result. Banded profiles produce many identical vectors, so a returned row often ties with the true neighbor; the recall function counts ties as correct, which is right, and also makes the task easy. Real vectors are rarely that tidy.
Three further traps are common. Queries that are also rows in the index make every search trivially find itself, which is why the recipe holds queries out. A mean over all queries hides the slice that matters, such as one tenant or a selective filter, where the post-filter plan above returned short results for half the queries. And distance computations measure work, not time: on a real engine, memory layout, caches and concurrency decide latency, so the final step of any recipe is p95 latency on the production engine at the expected query rate, as on scale.
In one breath#
A reference architecture is a set of decisions with reasons and measurements: storage layout, search algorithm and model chosen separately, filters placed inside the search, and a known place for every copy of the data. Mnemos keeps everything in one local file and searches exactly; the insurance build keeps both corpora in PostgreSQL with an index on one and an exact sort on the other; a managed Google Cloud stack keeps the corpora as tables and tests what the documentation leaves open; a Postgres-native service measures its plans in SQL. One recipe, an exact answer key with index recall, recall@k and distance computations, verifies all four.
The insurer can now defend a design in review with numbers instead of boxes. What remains is the code a team will write on the first day: the same retrieval as raw SQL, as direct SDK calls and inside the frameworks many teams start with, and what each abstraction hides. The next page, recipes, puts them side by side and closes with a decision guide for the whole site.