Module 18: Embeddings, Vector Databases & Semantic Search

Chapter Overview

Retrieval-augmented generation (RAG) has become the dominant pattern for grounding LLM outputs in factual, up-to-date information. At the foundation of every RAG system lies a trio of interconnected technologies: embedding models that convert text into dense vector representations, vector databases that store and index those vectors for efficient search, and document processing pipelines that prepare raw content for ingestion.

This module provides a comprehensive, bottom-up treatment of these foundational components. It begins with the theory and practice of text embedding models, covering the evolution from word-level embeddings to modern sentence transformers and the training objectives that produce high-quality representations. It then examines the data structures and algorithms that make approximate nearest neighbor search practical at scale, including HNSW graphs, inverted file indexes, and product quantization.

The module proceeds to survey the rapidly growing ecosystem of vector database systems, comparing purpose-built solutions like Pinecone, Weaviate, and Qdrant with library-based approaches such as FAISS and embedded databases like ChromaDB. Finally, it addresses the critical (and often overlooked) challenge of document processing and chunking, where poor design decisions can undermine even the most sophisticated retrieval infrastructure.

By the end of this module, you will be able to select and fine-tune embedding models for specific domains, design vector indexes that balance recall with latency, deploy and operate vector database systems in production, and build document processing pipelines that maximize retrieval quality.

Learning Objectives

Explain the evolution from word embeddings (Word2Vec, GloVe) to sentence-level embeddings (Sentence-BERT, E5, GTE)
Describe contrastive learning objectives and hard negative mining strategies for training embedding models
Compare cosine similarity, dot product, and Euclidean distance as vector similarity metrics
Explain HNSW graph construction and search, IVF partitioning, and Product Quantization at a technical level
Evaluate vector database systems (Pinecone, Weaviate, Qdrant, Milvus, ChromaDB, pgvector) for different deployment scenarios
Implement hybrid search combining dense vector retrieval with sparse keyword matching using reciprocal rank fusion
Design document chunking strategies (fixed-size, recursive, semantic, structure-aware) appropriate for different content types
Build end-to-end RAG ETL pipelines with document loading, parsing, chunking, embedding, and indexing
Use MTEB benchmarks and custom evaluation sets to select embedding models for specific domains
Implement incremental indexing and document versioning for production vector search systems

Prerequisites

Module 05: Embeddings and Representation Learning (vector spaces, similarity metrics)
Module 06: The Transformer Architecture (attention, encoder vs. decoder models)
Module 09: LLM APIs (working with embedding API endpoints)
Comfortable with Python, NumPy, and basic data structures (graphs, hash tables, trees)
Familiarity with database concepts (indexing, queries, CRUD operations)

Chapter Overview

Learning Objectives

Prerequisites

Sections