supraj.dev

Interview questions & answers

Vector Search

Embeddings, ANN indexes, filters, hybrid ranking and reindexing at scale.

  • 29 questions
  • 1 easy
  • 14 medium
  • 14 hard

29 questions

Question 1EasyEmbeddings

What is an embedding, and why does cosine similarity work for search?

#

A user searches "reset my password" and the right article is titled "Recovering account access".

Show answer

An embedding model turns text into a vector of numbers, trained so that texts with similar meaning get vectors pointing in similar directions. Cosine similarity measures the angle between two vectors, so it scores how close two meanings are regardless of vector length.

That is why "reset my password" can match "recovering account access" without sharing words. Embeddings are specific to the model that made them, so queries and documents must use the same model, and quality depends on how well the model fits your domain and language.

Trade-offs

Semantic matching handles paraphrases but can miss exact terms such as error codes.

Common mistake

Comparing vectors from two different embedding models.

Follow-up

When would dot product give different rankings from cosine similarity?

Question 2MediumIndexes

How does HNSW trade recall for speed, and which settings matter?

#

Search is fast, but a relevant document that exact search finds is often missing.

Show answer

HNSW builds a layered graph of vectors and searches it greedily from coarse to fine layers, which is much faster than comparing against every vector, at the cost of sometimes missing true neighbours.

The main settings: M, the number of links per node, which raises recall and memory; the construction search size (efConstruction), which improves graph quality at build time; and the query-time search size (ef_search in many systems, hnsw.ef_search in pgvector), which raises recall at the cost of latency. Measure recall against exact search on a sample of queries while tuning.

Trade-offs

Higher settings improve recall but use more memory, build time or query latency.

Common mistake

Tuning for speed without ever measuring recall against exact results.

Follow-up

How would you choose ef_search for a 50 ms latency budget?

Question 3MediumFiltering

Filtered vector search returns far fewer results than requested. Why?

#

Asking for the top 10 documents where tenant = acme returns only 2.

Show answer

Many indexes find the nearest neighbours first and apply the filter afterwards. If most candidates belong to other tenants, few survive the filter.

Options: use an engine that filters during the graph search; partition the index by tenant or other high-selectivity fields; raise the candidate count before filtering; or, in pgvector 0.8 and later, enable iterative index scans so the query keeps searching until enough rows pass the filter. For very selective filters, exact search over the filtered subset can be both faster and correct.

Trade-offs

Per-tenant partitions filter perfectly but multiply indexes to maintain.

Common mistake

Trusting top-k results from a post-filtered query without checking how many came back.

Follow-up

When would you choose exact search over an ANN index?

Question 4HardMigrations

You want a better embedding model in production. How do you switch?

#

The index holds 30 million chunks embedded with last year's model.

Show answer

Vectors from different models cannot be mixed, so you must re-embed the whole corpus. Build a new index alongside the old one, versioned by model, and backfill it in batches while new documents are written to both.

Evaluate retrieval on your labelled queries with both indexes before switching. Cut over by routing queries to the new index, ideally gradually, and keep the old one until the new one proves itself. Budget the embedding cost and time, and store the model name with every vector.

Trade-offs

Dual indexes double storage temporarily but make rollback simple.

Common mistake

Embedding new documents with the new model into the old index.

Follow-up

How would you keep both indexes consistent while documents are being deleted?

Question 5HardData lifecycle

A user asks to have their data deleted. How do you remove it from a vector index?

#

Their documents were chunked, embedded and cached in three places over the past year.

Show answer

You need to know which vectors came from which source: store document and owner IDs as metadata on every chunk at ingestion, so you can delete by filter. Delete from the primary store, the vector index, caches of retrieved results or answers, and any backups or exports under your retention policy.

Approximate indexes often mark deletions rather than removing them physically, and graph indexes can degrade with many deletes, so run the maintenance your store needs, such as vacuuming or rebuilding. Verify by searching for the user's content afterwards and record the deletion for audit.

Trade-offs

Rich metadata costs storage, but without it you cannot delete precisely.

Common mistake

Deleting the source documents and assuming their embeddings went with them.

Follow-up

How would you prove to an auditor that deletion completed everywhere?

Question 6HardScale

Your vector index needs 400 GB of RAM. How do you cut that?

#

You store 100 million 1,024-dimension float32 vectors.

Show answer

Raw vectors alone are 100M × 1,024 × 4 bytes ≈ 410 GB before index overhead. Quantization stores fewer bits per value: scalar quantization to int8 cuts it about four-fold, binary quantization much further, and product quantization compresses groups of dimensions into codes.

Shorter vectors also help: some models are trained so their leading dimensions work on their own (Matryoshka embeddings). Disk-based indexes keep most data on SSD. A common pattern is to search compressed vectors, then rescore the top candidates with full-precision vectors.

Trade-offs

Compression lowers recall; rescoring recovers much of it at extra latency.

Common mistake

Quantizing aggressively without measuring recall on real queries.

Follow-up

How would you size the rescoring step to stay within latency budget?

Question 7MediumChoice

pgvector or a dedicated vector database — how do you decide?

#

A team already runs PostgreSQL and expects 5 million documents next year.

Show answer

pgvector keeps vectors next to your relational data: transactions, joins, filters, backups and access control all work as usual, and there is no new system to run. For millions to tens of millions of vectors with a team that knows Postgres, it is usually the simplest good choice.

A dedicated vector database makes sense at much larger scale, for very high query rates, for advanced features such as built-in hybrid search or tiered storage, or when you want to scale search separately from the transactional database. Benchmark with your own data and filters.

Trade-offs

One database is simpler to operate, but heavy vector queries compete with transactional load.

Common mistake

Adopting a new database before measuring whether Postgres could handle the load.

Follow-up

Which metrics would tell you it is time to move off pgvector?

Question 8MediumIndexes

You created a pgvector IVFFlat index on an empty table during the first migration, then loaded 2 million rows. Recall is poor and does not improve when you add more data. What went wrong, and how would you set it up?

#

The index was created with the lists value from a tutorial. Queries use the default settings.

Show answer

IVFFlat splits vectors into lists around centroids that it computes from the data present when the index is built. Built on an empty or tiny table, the centroids do not represent the data you load later, so most vectors end up in a few lists and searches miss neighbours. The pgvector README says to "create the index after the table has some data".

Set it up properly: - Load (most of) the data first, then build the index. Rebuild it later if the data distribution shifts a lot. - Lists: the README suggests starting at rows / 1000 up to 1M rows and sqrt(rows) above 1M rows. For 2 million rows that is about 1,400 lists. - Probes: queries search 1 list by default. The README suggests starting at sqrt(lists), so about 37 here. Set it per session or transaction with SET ivfflat.probes = .... - Measure recall against exact search on a sample of real queries, and raise probes until recall meets your target at acceptable latency.

If data arrives continuously and you cannot pick a good moment to build, consider HNSW instead: it builds a graph rather than clusters, so there is no training step tied to the data present at build time, at the cost of slower builds and more memory.

Trade-offs

IVFFlat builds faster and uses less memory than HNSW but depends on the data at build time; HNSW adapts to inserts but costs more to build.

Common mistake

Creating an IVFFlat index in the schema migration before any data exists.

Follow-up

How would you decide when to rebuild an IVFFlat index as data grows?

Question 9MediumIndexes

A pgvector query with an HNSW index asks for LIMIT 100 but returns only 40 rows. There is no WHERE clause. Why, and what are your options?

#

pgvector 0.8 with default settings. The table has millions of rows.

Show answer

HNSW search keeps a dynamic candidate list of size hnsw.ef_search, and the README says results are limited by that list, which is 40 by default. Asking for 100 rows with ef_search at 40 gives at most 40.

Options: - Raise ef_search to at least the LIMIT, ideally with headroom: SET hnsw.ef_search = 200; per session or with SET LOCAL in a transaction. Higher values improve recall and cost latency. - If a filter is also involved, remember that with approximate indexes the filter is applied after the index scan, so even fewer rows survive. From 0.8.0 you can enable iterative index scans (hnsw.iterative_scan = strict_order or relaxed_order), which keep scanning until enough rows match, up to hnsw.max_scan_tuples (20,000 by default). - Ask whether you need 100 nearest rows at all. For RAG, retrieving 20 to 50 candidates and reranking is common.

Set ef_search deliberately per query type rather than relying on the default, and check recall against exact search when you change it.

Trade-offs

Higher ef_search returns more and better neighbours at higher latency; iterative scans fix filtered shortfalls but can visit many rows.

Common mistake

Assuming LIMIT alone controls how many neighbours an HNSW index search returns.

Follow-up

How would you choose ef_search for two query types with different latency budgets?

Question 10MediumIndexes

You built an HNSW index on a pgvector column, but EXPLAIN shows a sequential scan and queries take seconds. What would you check?

#

The index was created with vector_l2_ops. The query is SELECT ... ORDER BY 1 - (embedding <=> $1) DESC LIMIT 10.

Show answer

Two mismatches here.

- Operator class: the index was built with vector_l2_ops, which supports L2 distance (<->), but the query uses cosine distance (<=>). An index only serves the operator its operator class was built for. Build it with vector_cosine_ops for cosine, or query with the operator the index supports. - Query shape: the README says the query needs an ORDER BY and LIMIT, with the ORDER BY on the distance operator itself, ascending. ORDER BY 1 - (embedding <=> $1) DESC is an expression, so the planner cannot use the index. Write ORDER BY embedding <=> $1 LIMIT 10 and compute similarity in the select list if you need it.

Also check: - The query vector has the same type and dimensions as the column (for example vector(1024)), and expression indexes (such as a halfvec cast) are matched exactly by the query. - Small tables may get a sequential scan by choice; test with realistic data volumes. - Use EXPLAIN ANALYZE after each change to confirm an index scan.

Pick one distance for the product, matching what the embedding model was trained for, and build the index for it.

Trade-offs

One index per distance operator keeps queries fast but adds storage and build time if you support several.

Common mistake

Ordering by a derived similarity score instead of the raw distance operator.

Follow-up

How would you support both cosine and inner-product queries on the same table?

Question 11HardIndexes

You switched to a 3,072-dimension embedding model and pgvector refuses to build an HNSW index on the new vector(3072) column. How would you index it, and what do you check?

#

About 20 million chunks. Recall matters, and memory is limited.

Show answer

pgvector's README lists indexing limits by type: vector up to 2,000 dimensions, halfvec up to 4,000, bit up to 64,000 and sparsevec up to 1,000 non-zero elements. A 3,072-dimension vector column is over the limit.

Options: - Half precision: keep the column as is and build an expression index on a cast, as in the README's example pattern: CREATE INDEX ON items USING hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops);. Queries must use the same cast to hit the index. Half precision halves memory with usually small recall loss; measure it. - Binary quantization: index a bit version of the vector for a fast first pass, fetch more candidates than you need, and rerank them with full-precision distances. This saves far more memory but needs the rerank step to recover accuracy. - Shorter vectors: if the model supports truncated embeddings (some do), store fewer dimensions, which also cuts memory and latency.

Whichever you choose, measure recall against exact search on real queries, and check end-to-end retrieval quality, not only nearest-neighbour recall.

Trade-offs

halfvec is simple and close in accuracy; binary plus rerank saves the most memory but adds a query step; truncation depends on the model.

Common mistake

Writing queries without the same cast as the expression index, so the planner falls back to a full scan.

Follow-up

How many candidates would you fetch from a binary index before reranking, and how would you tune it?

Question 12MediumOperations

Building a pgvector HNSW index on 30 million rows has run for many hours. The log shows "hnsw graph no longer fits into maintenance_work_mem". How do you make the build practical?

#

A managed Postgres instance with 64 GB RAM. The index uses default parameters. Writes continue during the build.

Show answer

The notice means the graph outgrew maintenance_work_mem and the build switched to a much slower path; the README says the build "will take significantly more time" after that.

Make it practical: - Raise maintenance_work_mem for the build session so the graph fits, without exceeding what the server can spare (it is per operation, and a managed instance needs headroom for normal traffic). - Use parallel builds: pgvector uses parallel workers, 2 by default; raise max_parallel_maintenance_workers for the session. The README's example sets 7 plus the leader. - Build at a quiet time or on a replica you then promote, and use CREATE INDEX CONCURRENTLY if writes must continue, accepting a longer build. - Keep parameters deliberate: the defaults are m = 16 and ef_construction = 64. Higher values improve graph quality and recall but make builds slower and indexes larger. - Estimate memory first: vectors plus graph links, multiplied by row count. If it cannot fit, consider halfvec or binary quantization, or partitioning the table so each partition's index is smaller.

Trade-offs

More memory and workers speed up the build but compete with production queries on the same instance.

Common mistake

Leaving maintenance_work_mem at its default for a large vector index build and waiting.

Follow-up

How would you rebuild the index with new parameters without downtime?

Question 13HardOperations

Your pgvector table is updated constantly: embeddings are rewritten when documents change and old chunks are deleted. Over months, query latency rises and VACUUM on the table takes hours. What is happening and how would you manage it?

#

HNSW index, around 10 million live rows, several million updates a week.

Show answer

Each update in Postgres creates a new row version and a new index entry; the old ones stay until VACUUM removes them. For HNSW, cleaning dead entries means repairing the graph around them, which is expensive. The pgvector README warns that vacuuming can take a while for HNSW indexes. Meanwhile, dead entries still sit in the graph and make searches do more work.

Manage it: - Speed up vacuum as the README suggests: rebuild the index first with REINDEX INDEX CONCURRENTLY, then run VACUUM on the table. - Reduce churn: only re-embed when the text actually changed (compare a content hash), and avoid updating unrelated columns on the vector row; keep frequently changing metadata in a separate table joined by id. - Tune autovacuum for this table so dead rows are cleaned steadily rather than in huge batches. - For bulk re-embedding (for example a model change), build a new table or column and index, then switch, instead of updating every row in place. - Track dead tuples, index size and p95 latency over time so you rebuild before users notice.

Trade-offs

Periodic concurrent rebuilds keep the index healthy but cost CPU and I/O; splitting metadata from vectors adds joins but removes most churn.

Common mistake

Re-embedding every document on each sync even when its text has not changed.

Follow-up

How would you schedule index rebuilds so they never overlap peak traffic?

Question 14MediumChoice

A team wants to deploy a vector database cluster with HNSW for a feature that searches 50,000 help-centre chunks a few hundred times a day. What do you advise?

#

Embeddings have 768 dimensions. The data already lives in Postgres.

Show answer

You probably need no approximate index at all. Exact search over 50,000 vectors of 768 dimensions is about 38 million multiply-adds per query, which a single CPU core handles in milliseconds, and it gives perfect recall. The FAISS guidelines put it plainly: if you plan to perform only a few searches, direct computation is the most efficient option, and the flat index is the only one that can guarantee exact results.

Recommend: - Store the vectors in the existing Postgres with pgvector and run exact search without an index (or load them in memory in the service). No new system to run, back up and secure. - Measure latency at your real query rate; add an HNSW index only when data or traffic grows enough that exact search misses the latency target. - Spend the saved effort on retrieval quality: chunking, hybrid search for exact terms, and a reranker usually matter more than the index at this size.

Revisit when the corpus reaches millions of vectors or traffic grows to many queries per second.

Trade-offs

Exact search is simple and perfectly accurate but scales linearly with data; ANN indexes scale further at the cost of recall and operational work.

Common mistake

Choosing infrastructure for a scale the product does not have.

Follow-up

At what corpus size and query rate would you add an approximate index, and how would you decide?

Question 15HardIndexes

You must serve nearest-neighbour search over 500 million 768-dimension vectors from servers with 256 GB of RAM each, at high recall. Using FAISS, which index family would you start from and what would you tune?

#

Raw float32 vectors are about 1.5 TB. Some loss of recall is acceptable if it can be recovered by reranking.

Show answer

500M × 768 × 4 bytes is about 1.5 TB, so raw vectors and an HNSW graph will not fit in 256 GB. Use a compressed inverted-file index and rerank.

What the FAISS guidelines say: - For 100M to 1B vectors, use an IVF index with about 2^20 (1,048,576) centroids, using HNSW for cluster assignment, and train on 30 to 256 times as many vectors as centroids (training is slow at this size). - Compress with product quantization: PQ outputs M-byte codes per vector, so with M = 64 the codes take 32 GB for 500M vectors. OPQ adds a linear transformation that the guide describes as making vectors easier to compress. - Add a refinement stage that reranks the top candidates with more precise vectors; the guide notes this adds two parameters to tune. - HNSW alone would need about (d × 4 + M × 2 × 4) bytes per vector, roughly 3.2 KB at M = 16, which is why it does not fit here.

Tune: number of probes (lists searched), PQ size M, and how many candidates go to refinement, measuring recall against exact search on a sample. Keep the full-precision vectors on disk or in a separate store for refinement.

Check GPU support before planning on it: the guide lists IVF with flat or PQ as GPU-capable, but not HNSW or IVF with HNSW assignment.

Trade-offs

Heavier compression fits on fewer machines but lowers recall before reranking; more probes and refinement recover recall at query cost.

Common mistake

Training IVF centroids on a small or unrepresentative sample of a very large dataset.

Follow-up

How would you shard this index across machines, and what does each query then cost?

Question 16HardQuantization

To cut memory, you enabled binary quantization in Qdrant for 512-dimension embeddings. Memory fell sharply, but retrieval quality dropped badly. A colleague says binary quantization works fine at their company. What is different, and how do you recover quality?

#

Your colleague uses 3,072-dimension embeddings. You turned off rescoring to save latency.

Show answer

Binary quantization stores one bit per dimension, cutting vector memory by about 32 times. Qdrant's quantization guide says it is only efficient for high-dimensional vectors with a centred distribution of components, and that the binary pass alone loses accuracy which rescoring recovers. It gives both sides: text-embedding-3-large at 3,072 dimensions reached 0.9966 recall with 3x oversampling and rescoring, while 512-dimension text-embedding-3-small reached only 53% to 55% without rescoring and 71% to 91% with it.

So your setup combines the weak case (few dimensions) with no rescoring. To recover: - Turn rescoring back on and use oversampling: the guide's example is oversampling 2.4 with limit 100, which pre-selects 240 candidates from the quantized index and rescores them with the original vectors. Qdrant enables rescoring by default for binary quantization. - Try the richer options the guide describes for vectors under about a thousand dimensions: 2-bit (16x compression) or 1.5-bit (about 21x), or asymmetric quantization (binary stored vectors with scalar-quantized queries), which keeps binary-level memory with better precision. - Or use scalar int8 quantization, which gives 4x compression with minimal accuracy loss; its quantile setting (for example 0.99) clips outliers.

Keep original vectors on fast enough storage for rescoring, and measure recall on your data before and after.

Trade-offs

Binary quantization gives the biggest memory saving but needs high dimensions and rescoring; scalar int8 is safer at a quarter of the saving.

Common mistake

Disabling rescoring with binary quantization to save latency, without measuring recall.

Follow-up

How would you choose the oversampling factor for a fixed latency budget?

Question 17MediumQuantization

You recreated an Elasticsearch index on a new 9.x cluster with the same mapping, a dense_vector field of 1,024 dimensions with no index_options. Memory use dropped a lot and recall dropped slightly. Nobody changed any settings. Why?

#

The old cluster was on an earlier major version. The mapping does not specify index_options.

Show answer

The default index type changed. The dense_vector docs say that in 9.0 float vectors default to int8_hnsw, and from 9.1 vectors with 384 or more dimensions default to bbq_hnsw (fewer than 384 stay on int8_hnsw). BBQ is binary quantization combined with HNSW: one bit per dimension, about 32x less memory, at some cost in accuracy that oversampling and rescoring can largely offset.

What to do: - Decide explicitly. Put index_options in the mapping so behaviour does not change with upgrades. - If you keep BBQ, tune rescore_vector.oversample (valid values above 1.0 and below 10.0, or 0 to disable): higher values gather more candidates per shard to rescore with the raw vectors. - Size memory with the tuning guide's estimates: raw float vectors take dims × 4 bytes, int8 copies dims + 16, BBQ copies ceil(dims / 64) × 8 + 14. For 1,024 dimensions that is 4,096, 1,040 and 142 bytes per vector. The guide warns that once the working set no longer fits in cache, latency climbs sharply. - Measure recall against exact kNN on real queries for each option before choosing.

Trade-offs

BBQ saves most memory with some recall loss; int8 costs more memory but is closer to full precision.

Common mistake

Relying on mapping defaults across major upgrades and being surprised by a changed index type.

Follow-up

How would you run an A/B comparison of int8 and BBQ on live queries?

Question 18HardTuning

Your product team asks for "95% recall" from the vector index. How do you measure ANN recall in production, and how do you set search parameters to meet both that and a p99 latency target?

#

HNSW index, 40 million vectors, about 300 queries per second at peak. The corpus grows by about 5% a month.

Show answer

First agree on what recall means. Index recall is the share of the true k nearest neighbours (by exact distance) that the index returns. It is not relevance: a perfect index can still return useless chunks if the embeddings are poor. Measure both, separately.

Measuring index recall: - Sample a few thousand real queries from logs. - Compute exact top-k for each with brute force, offline, against a snapshot of the index data. - Run the same queries through the index and compute recall@k. - Repeat regularly (for example monthly, or after big data changes), because recall at fixed settings can drift as the data grows or its distribution shifts.

Tuning: the main query-time knob in HNSW is ef (hnswlib notes it cannot be lower than k; higher is more accurate but slower). Sweep ef and record recall and latency. Run the latency test at realistic concurrency, since p99 under load is much worse than single-query latency. Pick the lowest ef that meets recall, and check p99 at peak. If you cannot meet both, change something structural: more replicas, quantization with rescoring, filtering strategy, or a higher-quality graph (larger M or ef_construction, which needs a rebuild).

Alert on the recall check like any other SLO.

Trade-offs

Higher ef buys recall with latency and CPU; structural changes such as rebuilds or more replicas cost money and operational work.

Common mistake

Taking recall from a vendor benchmark instead of measuring it on your own data and queries.

Follow-up

How would you measure recall for filtered queries, where the ground truth depends on the filter?

Question 19MediumOperations

After bulk-loading 80 million documents with embeddings into Elasticsearch, kNN queries are much slower than in the proof of concept, even though the nodes have plenty of CPU. What would you look at?

#

Indexing continues at a high rate during the day. Documents store the vector in _source.

Show answer

Look at segments, memory and indexing load, in that order.

- Segments: a query searches the vector structures of every segment. The Elasticsearch tuning guide says kNN search can be up to several times faster with fewer segments. After a bulk load, force merge read-mostly indices, or create larger segments during bulk indexing. - Memory: the guide says data nodes need at least enough RAM to hold the vector data and index structures, off-heap, per copy plus headroom; once the working set no longer fits in cache, queries read from disk and latency climbs sharply. Estimate bytes per vector (dims × 4 for float, dims + 16 for int8, ceil(dims / 64) × 8 + 14 for BBQ) times document count times copies, and compare with free RAM. - Indexing: the guide notes that active indexing hurts approximate kNN performance, and suggests writing full reindexes to a separate index rather than updating in place. - _source: high-dimension vectors make _source large and expensive to load; exclude the vector field from _source if you do not need to return it. - Dimensions: search speed scales linearly with dimensions, so a smaller embedding model is another lever, after checking relevance.

Trade-offs

Force merging speeds search but is expensive and suits indices that are no longer written heavily; more RAM or quantization fixes cache misses at a cost.

Common mistake

Sizing a vector cluster by CPU and JVM heap while ignoring off-heap memory for the vectors.

Follow-up

How would you separate heavy indexing from search traffic in the same cluster?

Question 20HardMulti-tenancy

You are building a SaaS product with 50,000 tenants. Most have a few thousand vectors; a handful have tens of millions. How would you lay out tenants in a vector database?

#

Every query is scoped to one tenant. Large tenants pay for lower latency.

Show answer

Use a shared layout for the many small tenants and dedicated capacity for the few large ones.

With Qdrant as the example: - Collection per tenant is rarely efficient; each collection adds overhead, and Qdrant Cloud caps a cluster at 1,000 collections by default. Instead, put small tenants in one collection and filter by a tenant field. - Mark the tenant field's keyword payload index with is_tenant (from v1.11.0), so Qdrant stores each tenant's vectors together for sequential reads. - If queries are always per tenant, build per-tenant graphs instead of a global one: set payload_m (for example 16) and m = 0. The docs warn that m = 0 is not the same as exact search. - For large tenants, tiered multitenancy (v1.16.0) keeps small tenants in a shared fallback shard and moves big ones to dedicated shards; the docs suggest promoting at about 20,000 points and not creating more than a thousand dedicated shards per cluster.

Weaviate takes a different approach: each tenant gets its own shard, and tenants can be set INACTIVE (data on disk, not available) or OFFLOADED (moved to cloud storage), which suits many idle tenants. If you enable autoTenantCreation, watch for typos creating stray tenants.

Whatever the engine, enforce the tenant filter in the service layer, not in client code.

Trade-offs

Shared layouts are cheap for small tenants but share resources; dedicated shards isolate large tenants at higher cost and management effort.

Common mistake

Creating one collection per tenant and hitting limits and overhead at a few thousand tenants.

Follow-up

How would you move a tenant from the shared shard to a dedicated one without downtime?

Question 21HardFiltering

In Qdrant, filtered searches on "category" are slow and sometimes miss results, while unfiltered searches are fine. The payload index on category was added after all 30 million points were loaded. What is wrong and how would you fix it?

#

Some categories match millions of points, others only a few hundred.

Show answer

Two things go wrong when payload indexes come late.

- Filter-aware graph: Qdrant's indexing docs say the HNSW index only benefits from additional filter-aware edges when it is built after the payload indexes exist, and that payload indexes should be created before ingesting data. Without those edges, a restrictive filter can cut the graph into pieces the search cannot cross, which costs both speed and recall. - Query planning: Qdrant estimates how many points a filter matches and picks a strategy per segment: for small matches it retrieves points through the payload index and scores them directly; above a threshold it uses the filterable vector index; and for low selectivity with high cardinality it uses ACORN. The payload index is what lets it estimate cardinality, so without it the planner chooses badly.

Fix: - Create payload indexes for every field you filter on before loading data. For an existing collection, add them and then rebuild the vector index so it includes filter-aware edges (for example by recreating the collection with the payload indexes defined, then reloading). - Test both kinds of category: rare ones (should use the payload index path) and huge ones (should use the graph). Measure recall against exact filtered search. - Keep the set of filter fields stable; adding one later means another rebuild.

Trade-offs

Indexing filter fields up front makes ingestion slower and uses memory, but avoids rebuilds and keeps filtered recall high.

Common mistake

Treating payload indexes as an optimisation to add later rather than part of the schema.

Follow-up

How would you test filtered recall for a field with thousands of distinct values?

Question 22HardHybrid search

You combine BM25 and vector search by adding their scores. Results are dominated by whichever retriever happens to produce bigger numbers for a query. How should you fuse the two result lists?

#

BM25 scores range from 0 to about 30 depending on the query; cosine similarities are between 0 and 1.

Show answer

BM25 and vector similarity are on different, query-dependent scales, so raw addition lets one retriever dominate. Two standard fixes:

- Reciprocal rank fusion: ignore scores and combine ranks. Elasticsearch's RRF adds, for each retriever where a document appears, 1 / (k + rank), with ranks starting at 1 and k (rank_constant) defaulting to 60. A document ranked well by both lists rises to the top. The docs say RRF requires no tuning, and the input relevance scores do not need to be related to each other. Its window (rank_window_size, defaulting to the requested size) controls how deep each list is considered; too small, and good documents from one retriever never get a chance. - Score normalisation: OpenSearch's normalisation processor rescales each query's scores with min_max (the default), l2 or z_score, then combines them, for example with arithmetic_mean, which weights clauses equally unless you set weights. This keeps score magnitudes, which can help when one retriever is clearly more confident, but needs weight tuning.

Recommend RRF as the default, then test weighted normalisation on your labelled queries. Evaluate the fused ranking with recall and nDCG against both single retrievers, and look at queries with exact identifiers, where BM25 should win, and paraphrased questions, where vectors should.

Trade-offs

RRF is robust and tuning-free but discards score confidence; normalised score fusion keeps it but needs tuned weights per corpus.

Common mistake

Summing raw BM25 and cosine scores and tuning a weight on a handful of queries.

Follow-up

How would you add a cross-encoder reranker after fusion, and how many candidates would you pass to it?

Question 23MediumEmbeddings

Your index uses dot product. After adding vectors from a second embedding model, long documents keep ranking above more relevant short ones. The first model's vectors were fine. What is likely going on?

#

The first model's vectors were unit length. Nobody checked the second model's output.

Show answer

Dot product equals cosine similarity only when every vector has length 1. If the new model's vectors are not normalised, and their length grows with document length or some other feature, longer vectors score higher regardless of meaning.

The OpenAI embeddings guide shows why the first model worked: its embeddings are normalised to length 1, so cosine similarity can be computed with a dot product, and cosine and Euclidean distance give identical rankings. Elasticsearch's dot_product similarity states the requirement directly: for float vectors, all document and query vectors must be unit length. Its max_inner_product option exists for cases where magnitude should count.

Fix: - Normalise vectors from the new model at write time and at query time, or switch the index to cosine. - Never mix vectors from different models in one index anyway; even normalised, their spaces are not comparable. Keep a model version on each vector and query one model's index at a time. - Add a check in ingestion that rejects vectors with the wrong dimension or a norm outside the expected range. - Use the distance the model was trained for, as documented on its model card.

Trade-offs

Normalising everything makes metrics interchangeable but discards magnitude, which a few models use deliberately.

Common mistake

Assuming every embedding model returns unit-length vectors.

Follow-up

How would you migrate an index to a new model without a period of mixed vectors?

Question 24HardScale

Your vector index has outgrown RAM and the budget does not allow more memory. Someone suggests an SSD-based index. When does that work, and what do you need to watch?

#

About 1 billion vectors. Recall must stay high. Fast NVMe SSDs are available.

Show answer

SSD-resident graph indexes can work well if the design keeps random reads per query small. DiskANN is the reference design: its paper reports indexing a billion points on a workstation with 64 GB of RAM and an inexpensive SSD, serving more than 5,000 queries per second on 16 cores with mean latency under 3 ms at 95%+ 1-recall@1. It contrasts this with compression-based methods of similar memory footprint that plateau around 50% 1-recall@1, and says it can serve 5 to 10 times more points per node than in-memory graph indexes in the high-recall regime.

The idea: keep compressed vectors in RAM to guide the graph search, keep the graph and full vectors on SSD, and read only what the search visits.

Watch: - SSD random-read performance and queue depth at peak concurrency; cheap cloud disks with low IOPS will not behave like local NVMe. - Tail latency, which depends on how many SSD reads a query makes; tune the search list size and measure p99, not mean. - Updates: many disk-based indexes are optimised for mostly static data; check how your engine handles inserts and deletes. - Page cache: engines that rely on the OS cache slow down sharply once the hot set no longer fits, as Elasticsearch's tuning guide warns.

Engines increasingly offer disk-oriented index types (Elasticsearch lists bbq_disk as a default from 9.4 where the licence allows); benchmark on your data before committing.

Trade-offs

SSD indexes cut memory cost per vector a lot, at the price of higher and more variable latency and dependence on disk performance.

Common mistake

Testing an SSD index on local NVMe and deploying on network storage with far lower IOPS.

Follow-up

How would you size the in-memory compressed vectors for a disk-based index?

Question 25HardCost

Estimate the memory needed to serve 200 million 1,024-dimension vectors with HNSW, two copies for availability, and show how quantization changes the answer.

#

You need the estimate to compare a managed vector service against self-hosting. Use M = 16.

Show answer

Work it out per vector, then scale.

- Raw float32 vectors: 1,024 × 4 = 4,096 bytes. For 200M vectors that is about 819 GB. - HNSW links: the FAISS guidelines estimate HNSW memory as (d × 4 + M × 2 × 4) bytes per vector, so the link part is 16 × 2 × 4 = 128 bytes, about 26 GB for 200M. (hnswlib gives a similar rule of thumb, roughly M × 8 to 10 bytes per element.) - Total at full precision: about 845 GB per copy, about 1.7 TB for two copies, before engine overhead and headroom.

With quantization, using Elasticsearch's published per-vector estimates: - int8: dims + 16 = 1,040 bytes, about 208 GB for the vectors. - BBQ (binary): ceil(1,024 / 64) × 8 + 14 = 142 bytes, about 28 GB. Add the graph links and multiply by two copies. Quantized setups usually keep full vectors on disk for rescoring, so they need disk space and some I/O, not RAM.

So with links and two copies, the same workload needs roughly 1.7 TB of RAM at full precision, about 470 GB with int8 and about 110 GB with BBQ. Then price it: RAM-heavy instances for each option, plus the managed service's quote for the same recall and latency targets, measured on your data. Add headroom for growth and for rebuilds, which can need a second copy of the index temporarily.

Trade-offs

Quantization cuts the largest cost driver by 4x to 30x but costs some recall and adds rescoring work.

Common mistake

Estimating from raw vector size alone and forgetting graph links, replicas and rebuild headroom.

Follow-up

How does the estimate change if you shorten vectors to 512 dimensions instead of quantizing?

Question 26MediumScale

Your vector search cluster misses its p99 latency target at peak. One engineer wants more shards; another wants more replicas. How do you decide?

#

The index fits in memory across the current nodes. CPU is high on all nodes at peak. Each query hits every shard.

Show answer

They fix different problems.

- Shards split the data. Each query fans out to every shard and waits for the slowest one, so more shards mean more work per query and more chances of a slow straggler. They help when data no longer fits on the nodes, or when each shard's search is too slow because it is too big. - Replicas copy the data. Each query goes to one copy of each shard, so replicas spread queries across more machines and raise throughput. They help when nodes are busy because of query volume.

Here the index fits and CPU is high everywhere at peak, which points to throughput: add replicas (and the nodes to hold them). More shards would add fan-out overhead without reducing total work.

Also check: - Whether queries could avoid the fan-out: if every query is scoped to one tenant, route by tenant to a single shard (custom sharding), which cuts work per query. - Search parameters: an ef or candidate count higher than needed costs CPU on every shard. - Tail latency from garbage collection, segment merges or indexing on the same nodes.

Measure p99 per shard to find stragglers before scaling either way.

Trade-offs

Replicas raise throughput and availability at the cost of memory for each copy; shards raise capacity but add fan-out latency.

Common mistake

Adding shards to fix a throughput problem, which increases per-query fan-out.

Follow-up

How would you route queries to a tenant's shard while keeping small tenants on shared shards?

Question 27MediumConsistency

A user uploads a document and immediately asks a question about it, but retrieval does not find it. Ten seconds later it works. How would you explain this, and how would you fix the experience?

#

The vector store is Milvus with default settings. Uploads and queries come through the same backend service.

Show answer

The default consistency does not guarantee that a search sees writes made a moment ago. Milvus documents four levels: Strong (always sees the latest data, highest latency), Bounded staleness (the default; recent writes may be briefly invisible), Session (writes from the same session are visible in that session) and Eventually (fastest, no ordering guarantee). It implements them with a guarantee timestamp that tells query nodes how up to date their data must be before serving the request.

Fixes, from narrowest to broadest: - Use Session consistency for the user who just uploaded, so their own writes are visible to them, while other traffic stays on the default. - Use Strong consistency only for the first query after an upload, if the client library lets you set it per request. - In the product, show the document as "processing" until indexing is confirmed, and answer from the uploaded text directly for the first question.

Other engines have similar trade-offs: Qdrant, for example, has an indexed_only search option that skips segments whose index is not built yet, which can return partial results. Know your engine's behaviour and test the upload-then-ask path end to end.

Trade-offs

Stronger consistency gives fresher results at higher latency; weaker levels are faster but can miss recent writes.

Common mistake

Switching the whole system to Strong consistency to fix one user flow.

Follow-up

How would you write an automated test that catches this race?

Question 28HardRetrieval models

Single-vector retrieval misses passages where one rare term matters, and a cross-encoder reranker over many candidates is too slow. Someone suggests ColBERT-style late interaction. How does it work, and what does it cost to store and serve?

#

Technical documentation with many product codes and parameter names. About 5 million passages.

Show answer

Late interaction encodes the query and the document separately, as in single-vector retrieval, but keeps one vector per token. At query time, each query token finds its most similar document token, and those maximum similarities are summed (MaxSim). Rare terms keep their own vectors, so they are not averaged away. The ColBERT paper reports quality competitive with BERT-based rerankers while running two orders of magnitude faster and needing four orders of magnitude fewer FLOPs per query, because document representations are computed offline.

Costs: - Storage: one vector per token, not per passage, so tens to hundreds of times more vectors. This is the main cost; compression and pooling matter. - Engine support: Qdrant stores multivectors per point with a max_sim comparator, with a hard limit of number_of_sub_vectors × vector_size below 1,048,576; the docs suggest pooling token vectors or splitting documents if you exceed it.

A practical design: retrieve candidates with a cheap single-vector or hybrid search, then rescore the top few hundred with late interaction, instead of indexing every token vector for first-stage search. That gets most of the precision on rare terms at a fraction of the storage and latency. Compare against hybrid BM25 plus a dense vector, which may already fix the product-code misses more cheaply.

Trade-offs

Late interaction improves term-level precision over single vectors and is cheaper than cross-encoders, but multiplies storage.

Common mistake

Indexing every token vector for first-stage search without estimating storage.

Follow-up

How would you measure whether late interaction beats hybrid search plus a reranker on your data?

Question 29MediumChoice

Three vector database vendors each show benchmarks where they are fastest. How would you run your own evaluation to choose between them?

#

Your workload: 30 million vectors, tenant filters on every query, steady inserts and deletes, a p99 target of 100 ms.

Show answer

Vendor benchmarks usually measure the case each vendor is best at: a static public dataset, no filters, no concurrent writes, and a recall level that suits them. Test your own case.

Design: - Data: a sample of your own vectors (same model and dimensions), at realistic scale, ideally the full 30 million. - Queries: real queries from logs with their tenant filters, including small and large tenants. - Ground truth: exact filtered top-k for each query, computed offline, so you can measure recall. - Compare at equal recall: for each engine, tune parameters to reach, say, 95% recall@10, then compare latency and throughput. Comparing speed at different recall levels is meaningless. - Load: run at peak concurrency with inserts and deletes happening, and record p50, p99 and errors over at least an hour, including during compaction or index rebuilds. - Operations: backup and restore time, rebuild time after a parameter change, upgrade process, monitoring, and access control. - Cost: the hardware or service tier each needed to meet the target, at your growth rate.

Write the criteria and weights before running the tests, so the results decide rather than the most persuasive vendor.

Trade-offs

An evaluation on your data takes a few weeks but avoids an expensive migration later; vendor numbers are free but rarely transfer.

Common mistake

Comparing query speed across engines without fixing recall, filters and write load.

Follow-up

How would you include managed services, where you cannot control the hardware, in the same comparison?