TOPIC AI
Embeddings converting text into numerical vectors for semantic search
IN 10 SECONDS
An embedding model converts text — a word, sentence, or document — into a fixed-length array of floating-point numbers (e.g., 1536 dimensions for `text-embedding-3-small`). The key property: semantically similar texts produce vectors that are close together in vector space (high cosine similarity). This is the foundation of RAG, semantic search, and clustering.
GOTCHA Embedding models have a maximum input token limit (usually 512-8192 tokens). Text longer than the limit is silently truncated, losing information. Chunk your documents into segments smaller than the limit before embedding. Also, different embedding models produce incompatible vector spaces — you can't mix them.
HOW EMBEDDINGS ARE CREATED AND USED
01 Input text 'What is Kubernetes?' is sent to an embedding model API.
02 Embedding model processes the text through its transformer layers and outputs a 1536-dimensional vector.
03 Vector index the vector is stored in a vector DB (pgvector, Pinecone, Qdrant) alongside the original text.
04 Semantic search a query is embedded with the same model. The vector DB returns the nearest neighbors by cosine distance.
POKE IT YOURSELF
curl -X POST -d '{"input":"What is Kubernetes?","model":"text-embedding-3-small"}' https://api.openai.com/v1/embeddings — generate an embedding
python -c "from sentence_transformers import SentenceTransformer; model = SentenceTransformer('all-MiniLM-L6-v2'); emb = model.encode('Hello world')" — generate embeddings locally
Drill this topic →
~75 sec read