Vector DB Comparison

MLOps Series RAG Systems

Benchmark Methodology

A fair comparison requires a controlled setup. Our benchmarks use the ann-benchmarks framework with the following parameters: 1M vectors of dimension 1536 (matching OpenAI text-embedding-3-small output), cosine similarity metric, recall@10 target of ≥0.95, and a single-node deployment for each database on an r6i.2xlarge EC2 instance (8 vCPU, 64 GB RAM, gp3 SSD).

Each database was configured with its recommended HNSW parameters: M=16, ef_construction=200, ef_search=128 (tuned to hit the 0.95 recall target). For Milvus, we additionally tested IVF-PQ as a memory-optimised alternative. Chroma was tested in persistent mode with its default HNSW settings. Indexing was timed end-to-end from the first upsert call to the last flush confirmation.

Disclaimer: Benchmarks are highly sensitive to hardware, dataset distribution, and configuration tuning. Treat these as directional rather than absolute.

Performance Results

Query Latency (p50 / p99)

Database	p50 Latency (ms)	p99 Latency (ms)	QPS (single thread)	QPS (32 threads)	Recall@10
Pinecone (s2)	8	22	120	2,800	0.96
Weaviate 1.26	3	11	310	6,500	0.95
Qdrant 1.10	2	7	480	9,200	0.96
Milvus 2.4 (HNSW)	3	14	350	7,800	0.95
Milvus 2.4 (IVF-PQ)	5	18	210	4,200	0.93
Chroma 0.5	12	45	80	950	0.95

Qdrant leads in raw latency and throughput, benefiting from its Rust implementation and in-graph filtering. Weaviate and Milvus-HNSW are close behind. Chroma is significantly slower at scale due to its Python coordination layer, but perfectly adequate for sub-100K collections.

Index Build Time

Database	1M Vectors (min)	Peak RAM (GB)	Disk Usage (GB)
Pinecone	~14 (API upsert)	Managed	Managed
Weaviate	8.2	18.4	9.1
Qdrant	6.5	14.8	7.6
Milvus (HNSW)	9.1	21.2	10.3
Chroma	22.0	24.6	11.2

Feature Matrix

Feature	Pinecone	Weaviate	Qdrant	Milvus	Chroma
Open Source	No	Yes	Yes	Yes	Yes
HNSW Index	Yes*	Yes	Yes	Yes	Yes
IVF-PQ Index	No	No	No	Yes	No
DiskANN	No	No	No	Yes	No
Scalar Quantisation	No	No	Yes	No	No
Hybrid Search	No	Yes	Sparse vecs	Yes	No
Multi-tenancy	Namespaces	Native	Collections	Partitions	Collections
Metadata Filtering	JSON	GraphQL	Payload idx	Expr	Where dict
Built-in Vectoriser	No	Yes	No	No	Plugin
Replication	Managed	Raft	Raft	Yes	No
GPU Acceleration	No	No	No	GPU-IVF	No

* Pinecone does not expose index internals; the underlying structure is proprietary.

Cost Analysis

Cost modelling depends on vector count, query volume, and whether you use managed or self-hosted infrastructure. Below is a simplified monthly cost estimate for a 5M-vector index serving 100 QPS.

Managed Services

Pinecone s2: ~$280/mo (pod-based) or usage-based on serverless
Weaviate Cloud: ~$220/mo (performance tier, 5M vectors)
Qdrant Cloud: ~$195/mo (4 GB RAM node, 5M vectors)
Zilliz Cloud: ~$250/mo (performance CU, 5M vectors)

Self-Hosted (AWS)

EC2 r6i.xlarge: ~$145/mo (on-demand) + EBS
EKS overhead: ~$73/mo (control plane)
Total infra: ~$230/mo before ops labour
Break-even: Self-hosted wins past ~20M vectors

Key insight: At ≤5M vectors and ≤200 QPS, managed services are almost always cheaper than self-hosted when you factor in engineering time for upgrades, monitoring, and incident response.

Scalability

Scalability encompasses three axes: vector count (how many vectors fit in a single node before you must shard), query throughput (horizontal read scaling), and write throughput (ingestion rate during bulk loads or streaming updates).

Database	Max Vectors / Node	Sharding	Read Replicas	Streaming Ingest
Pinecone	Managed (unlimited)	Auto	Auto	Yes
Weaviate	~50M (64 GB RAM)	Hash-based	Raft replicas	Yes (batch API)
Qdrant	~80M (64 GB RAM, SQ)	Hash-based	Raft replicas	Yes
Milvus	~30M (64 GB, HNSW)	Channel-based	Query nodes	Yes (WAL)
Chroma	~5M (64 GB RAM)	None (single)	None	Yes (batch)

For billion-scale deployments, Milvus and Qdrant are the strongest contenders. Milvus's disaggregated architecture lets you scale query nodes independently from data nodes, while Qdrant's quantisation and memory-mapped segments push single-node capacity higher than competitors with similar RAM.

Recommendations

There is no universally "best" vector database — the right choice depends on your constraints. Here is a decision framework based on common scenarios:

# Decision tree (pseudo-code) if vectors < 100_000 and team == "prototyping": return "Chroma" # zero-config, pip install, in-process elif need_zero_ops and budget_ok: return "Pinecone Serverless" # managed, pay-per-use elif need_hybrid_search: return "Weaviate" # best BM25 + dense fusion elif vectors > 50_000_000 and need_gpu_index: return "Milvus" # GPU-IVF, disaggregated arch elif latency_critical and filtered_search: return "Qdrant" # Rust speed, in-graph filtering else: evaluate("Weaviate", "Qdrant") # both are strong defaults

Pro tip: Abstract your vector DB behind a thin interface (upsert, query, delete). This lets you swap backends without rewriting application logic — critical when the ecosystem is evolving this fast.

Abstraction Layer Example

from abc import ABC, abstractmethod from typing import Any class VectorStore(ABC): """Thin abstraction over any vector database.""" @abstractmethod def upsert(self, ids: list[str], vectors: list[list[float]], metadatas: list[dict]) -> None: ... @abstractmethod def query(self, vector: list[float], top_k: int = 10, filters: dict | None = None) -> list[dict]: ... @abstractmethod def delete(self, ids: list[str]) -> None: ... class QdrantStore(VectorStore): def __init__(self, url, collection): from qdrant_client import QdrantClient self.client = QdrantClient(url=url) self.collection = collection def upsert(self, ids, vectors, metadatas): from qdrant_client.models import PointStruct points = [ PointStruct(id=i, vector=v, payload=m) for i, v, m in zip(ids, vectors, metadatas) ] self.client.upsert(self.collection, points=points) def query(self, vector, top_k=10, filters=None): hits = self.client.query_points( self.collection, query=vector, limit=top_k ) return [{"id": h.id, "score": h.score, **h.payload} for h in hits.points] def delete(self, ids): self.client.delete(self.collection, points_selector=ids)

Next: Understand how to split documents before indexing in Chunking Strategies.