Embedding
Definition
A dense numerical vector representation of text (or other data) that captures semantic meaning in a high-dimensional space. Semantically similar texts produce vectors that are close together (measured by cosine similarity or dot product). Embeddings are generated by specialized models (OpenAI text-embedding-3, Cohere embed, open-source models like nomic-embed) and are the foundation of semantic search, RAG, clustering, and classification in agent systems. Embedding quality — how well the vectors capture the nuances of your specific domain — directly determines the quality of downstream tasks.
Builder Context
Embedding model choice matters less than you think; chunking strategy and query formulation matter more. That said: use the latest model from your provider (embedding models improve faster than people switch), match your embedding dimensions to your latency requirements (smaller dimensions = faster search, slightly less accuracy), and always normalize vectors before storing. For domain-specific applications, fine-tuning an embedding model on your data can improve retrieval quality by 10-30%, but only after you've optimized chunking and query formulation.