Embedding Models: OpenAI, Cohere, BGE, E5

What Are Embeddings?

An embedding is a fixed-length dense vector in ℝ^d that captures semantic meaning. Two texts about the same concept yield vectors with high cosine similarity, even when they share few lexical tokens. Modern embedding models are trained via contrastive learning — positive pairs (query, relevant passage) are pulled together while negatives are pushed apart — producing representations that excel at nearest-neighbor retrieval tasks.

The choice of embedding model directly determines RAG retrieval quality. A poor encoder that conflates unrelated concepts will surface irrelevant chunks, forcing the LLM to hallucinate. Conversely, a high-quality encoder with strong domain alignment makes even a simple top-k retrieval strategy competitive with elaborate re-ranking pipelines.

OpenAI Embeddings

OpenAI offers two production embedding models: text-embedding-3-small (1536 dimensions, ~$0.02/1M tokens) and text-embedding-3-large (3072 dimensions, ~$0.13/1M tokens). Both support Matryoshka Representation Learning (MRL) — you can request shorter vectors (e.g., 256 or 512 dims) via the dimensions parameter, trading a small accuracy drop for significant storage and latency savings.

The v3 family improved over text-embedding-ada-002 across all MTEB benchmarks, with 3-large achieving a MTEB average of 64.6 versus ada-002's 61.0. The API is straightforward — a single HTTP call returns a list of float vectors, one per input text. Rate limits and token counts apply per-request.

Cohere Embed v3

Cohere's embed-english-v3.0 and embed-multilingual-v3.0 output 1024-dimensional vectors and natively support input types — you specify whether the input is a search_document or search_query, which triggers asymmetric encoding that boosts retrieval accuracy. Cohere reports that v3 achieves state-of-the-art results on BEIR, outperforming OpenAI's ada-002 on 11 of 15 datasets.

A distinguishing feature is the built-in compression options. You can request int8 or binary quantisation directly from the API, reducing storage by 4× or 32× respectively. Binary embeddings enable blazing-fast Hamming-distance search in memory-constrained environments, with a full-precision re-ranking pass on the shortlist.

Open-Source: BGE & E5

For teams that need on-premise deployment or want to avoid per-token API costs, BGE (BAAI General Embedding) and E5 (Microsoft) are the leading open-source options. Both are available in small (~33M params, 384 dims), base (~110M, 768 dims), and large (~335M, 1024 dims) variants on Hugging Face.

BGE-large-en-v1.5 holds a top position on the MTEB leaderboard among open models. It was trained with RetroMAE pre-training followed by contrastive fine-tuning with hard negatives mined from cross-encoder scores. E5-mistral-7b-instruct pushes the envelope further by adapting a full LLM as an encoder, achieving parity with some commercial APIs at the cost of higher inference compute.

Running these models locally with sentence-transformers or ONNX Runtime gives you full control over batching, quantisation (INT8 / FP16), and hardware placement (GPU vs CPU). For large-scale offline indexing jobs, this eliminates API rate-limit bottlenecks entirely.

Choosing a Model

When selecting a model, consider: (1) latency budget — API calls add network round-trip; local models add GPU inference time; (2) dimensionality — higher dims improve recall but increase vector DB storage and search cost; (3) domain alignment — run your own eval set on BEIR or a custom relevance benchmark before committing.

Code Examples

OpenAI

Cohere

BGE / E5 (Local)