AI / A CONCEPT NOTE
Vector Databases
storing and searching by semantic similarity, not exact keywords
Overview · mechanism
pitfall · examples
01 / THE SHORT VERSION
The idea in a few sentences.
A vector database (Pinecone, Weaviate, Qdrant, pgvector) stores high-dimensional embeddings alongside metadata. Instead of WHERE title LIKE '%rag%', you search by cosine distance: ORDER BY embedding <=> query_embedding LIMIT 5. This finds documents that are semantically similar, even if they share no exact keywords.
02 / FOLLOW THE MECHANISM
How a vector search works
Ingestion
each document chunk is passed through an embedding model to produce a vector (e.g., 1536 floats for
text-embedding-3-small).Indexing
the vector + metadata + document text are stored. An approximate nearest neighbor (ANN) index like HNSW is built for fast search.
Query
the user's query is embedded with the same model, producing a vector in the same latent space.
Search
the index returns the top-k nearest neighbors by cosine similarity or Euclidean distance, along with their stored metadata.
04 / COMMAND NOTES
Read the command, then the result.
Inspect the flags and arguments before trying an example. Snippets can need local setup, replacement values, or resources in your own environment.
search via Qdrant API
curl -X POST -d '{"vector":[0.1,0.2,...],"topK":5}' http://localhost:6333/collections/my-collection/points/searchThe ellipsis omits required code or values. This sketch is not runnable as written.
05 / CHECK YOURSELF
Could you explain Vector Databases to a teammate?
Try it out loud in two sentences: what it is, and the one detail that changes the picture. If you stall, the gap is the part to reread.
Up next in AI engineeringPrompt Engineeringcrafting inputs to get reliable, structured outputs from LLMs