Vector DBs: Pinecone, Weaviate, Qdrant, Milvus, Chroma

MLOps Series RAG Systems

Why Vector Databases?

Traditional relational databases optimise for exact-match lookups on structured fields. Semantic search requires a fundamentally different operation: given a high-dimensional float vector, find the k nearest neighbors by cosine similarity or inner product. Brute-force comparison scales as O(n·d), which becomes intractable beyond a few hundred thousand vectors.

Vector databases solve this with approximate nearest-neighbor (ANN) indexes — data structures like HNSW (hierarchical navigable small-world graphs), IVF (inverted file with product quantisation), and ScaNN that reduce query complexity to sub-linear time. They also manage metadata filtering, persistence, replication, and CRUD operations — turning ANN search into a production-grade data service.

Core idea: A vector DB = ANN index + metadata store + CRUD API + persistence + (optionally) replication.

Pinecone

Pinecone is a fully managed, serverless vector database. You create an index specifying dimension and metric (cosine, dotproduct, euclidean), then upsert vectors with JSON metadata. Queries support top-k retrieval with optional metadata filters expressed as JSON predicates. Pinecone manages sharding, replication, and scaling automatically — there are no knobs for index type or segment tuning.

Pinecone's serverless tier (launched 2024) bills per read/write unit rather than per pod-hour, making it cost-effective for bursty workloads. For steady high-throughput scenarios, pod-based indexes offer predictable performance. The main trade-off is vendor lock-in — there is no self-hosted option and no way to export the raw index structure.

from pinecone import Pinecone, ServerlessSpec pc = Pinecone(api_key="YOUR_KEY") # Create a serverless index pc.create_index( name="rag-docs", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1") ) idx = pc.Index("rag-docs") # Upsert vectors with metadata idx.upsert(vectors=[ {"id": "doc-1", "values": embed("RAG overview..."), "metadata": {"source": "wiki", "topic": "rag"}} ]) # Query with metadata filter results = idx.query( vector=embed("What is RAG?"), top_k=5, include_metadata=True, filter={"topic": {"$eq": "rag"}} )

Weaviate

Weaviate is an open-source vector database written in Go with a GraphQL / REST / gRPC API. It supports modules that integrate embedding models directly into the DB — when you upsert a text object, Weaviate calls the configured vectoriser (OpenAI, Cohere, or a local Hugging Face model) server-side, so you never manage embeddings in application code.

Under the hood, Weaviate uses a custom HNSW implementation with flat-file storage. It supports hybrid search (dense + BM25 keyword) via the alpha parameter, letting you blend semantic and lexical relevance. Multi-tenancy is first-class: each tenant's data is stored in isolated shards, useful for SaaS applications where customer data must be segregated.

import weaviate from weaviate.classes.config import Configure, Property, DataType client = weaviate.connect_to_local() # Define a collection with a vectoriser module docs = client.collections.create( name="Document", vectorizer_config=Configure.Vectorizer.text2vec_openai(), properties=[ Property(name="content", data_type=DataType.TEXT), Property(name="source", data_type=DataType.TEXT), ] ) # Insert — Weaviate auto-embeds the "content" field docs.data.insert({"content": "RAG pipelines retrieve...", "source": "blog"}) # Hybrid search (dense + BM25) results = docs.query.hybrid( query="vector database architecture", alpha=0.75, # 0=pure keyword, 1=pure vector limit=5 )

Qdrant

Qdrant is a Rust-based vector search engine built for high-throughput, low-latency retrieval. Its standout features include named vectors (store multiple embedding representations per point), payload indexes (secondary indexes on metadata fields for fast filtered search), and quantisation (scalar and product quantisation built into the engine for memory savings up to 4×).

Qdrant exposes a gRPC + REST API and ships as a single Docker image. In distributed mode, it uses Raft consensus for collection-level metadata and shard-level replication for data durability. The filtering engine is integrated into the HNSW traversal itself — rather than post-filtering top-k results, Qdrant prunes the graph during search, producing correctly filtered results without recall loss.

from qdrant_client import QdrantClient from qdrant_client.models import ( VectorParams, Distance, PointStruct, Filter, FieldCondition, MatchValue ) qd = QdrantClient(url="http://localhost:6333") # Create collection with HNSW + cosine qd.create_collection( collection_name="docs", vectors_config=VectorParams(size=1536, distance=Distance.COSINE) ) # Upsert points qd.upsert(collection_name="docs", points=[ PointStruct(id=1, vector=embed("RAG overview..."), payload={"source": "wiki"}) ]) # Filtered search hits = qd.query_points( collection_name="docs", query=embed("What is RAG?"), query_filter=Filter(must=[ FieldCondition(key="source", match=MatchValue(value="wiki")) ]), limit=5 )

Milvus

Milvus is a cloud-native, distributed vector database originally developed at Zilliz. It separates compute and storage, with a query-node layer for search and a data-node layer for ingestion. This disaggregated architecture lets you scale reads and writes independently — ideal for workloads where indexing throughput and query latency have different scaling curves.

Milvus supports a wide range of index types: HNSW, IVF-FLAT, IVF-PQ, IVF-SQ8, DiskANN, and GPU-IVF. It also offers a hybrid search API that combines dense and sparse (BM25) vectors in a single query, with RRF (reciprocal rank fusion) or weighted scoring for result merging.

from pymilvus import MilvusClient mc = MilvusClient(uri="http://localhost:19530") # Create collection with auto-ID and dynamic schema mc.create_collection( collection_name="docs", dimension=1536, metric_type="COSINE", auto_id=True ) # Insert data mc.insert( collection_name="docs", data=[{"vector": embed("RAG overview..."), "source": "wiki"}] ) # Search with filter expression results = mc.search( collection_name="docs", data=[embed("What is RAG?")], limit=5, output_fields=["source"], filter='source == "wiki"' )

Chroma

Chroma bills itself as the "AI-native open-source embedding database." It is written in Python with a Rust-based storage backend and targets the prototyping-to-production workflow. A single pip install chromadb gives you an in-process database — no Docker, no server — that persists to disk via SQLite + Arrow files. For production, Chroma offers a client-server mode and a hosted cloud offering.

Chroma's API is intentionally minimal: collections hold documents, embeddings, and metadata. It supports pluggable embedding functions — pass an OpenAI or Sentence-Transformers encoder at collection creation, and Chroma handles encoding transparently on upsert and query.

import chromadb from chromadb.utils import embedding_functions # Ephemeral (in-memory) or PersistentClient for disk client = chromadb.PersistentClient(path="./chroma_data") ef = embedding_functions.OpenAIEmbeddingFunction( model_name="text-embedding-3-small" ) col = client.get_or_create_collection(name="docs", embedding_function=ef) # Upsert — Chroma auto-embeds the documents col.upsert( documents=["RAG pipelines retrieve context..."], metadatas=[{"source": "blog"}], ids=["doc-1"] ) # Query results = col.query( query_texts=["What is retrieval-augmented generation?"], n_results=5, where={"source": "blog"} )

Hosted vs Self-Hosted

Managed / Hosted

Zero ops — provider handles scaling, backups, upgrades
Pay-per-use or pod-hour pricing
Higher per-query cost at large scale
Data leaves your VPC (unless private-link option)
Examples: Pinecone, Weaviate Cloud, Qdrant Cloud, Zilliz Cloud

Self-Hosted

Full control — tune HNSW params, segment sizes, replicas
Compute cost only (Kubernetes, bare metal)
Lower marginal cost at high vector counts
Data stays in your infrastructure
Examples: Weaviate OSS, Qdrant OSS, Milvus, Chroma

For early-stage projects, start with Chroma (zero infrastructure) or Pinecone serverless (managed, pay per use). As you scale past 10M vectors or need fine-grained index tuning, migrate to Qdrant or Milvus on Kubernetes where you control memory allocation, replication factor, and HNSW build parameters.

Next: See Vector DB Comparison for head-to-head benchmarks across these five databases.