Bi-Encoder

An architecture where two input texts are encoded separately into dense vectors and then compared (e.g., using cosine similarity), which is much faster than cross-encoders and powers vector databases.

How it works

A bi-encoder passes each text independently through a Transformer encoder and pools the output into a single dense vector (typically via mean pooling or [CLS] token). Similarity between two texts is then computed as the cosine or dot product of their vectors. Because the encoding is independent, all documents can be encoded offline and stored in a vector database. At query time, only the query is encoded, and fast ANN search finds the nearest stored vectors.

Why it matters

Bi-encoders are the foundation of semantic search and retrieval-augmented generation at scale. Unlike cross-encoders, document encoding is done once offline, making real-time retrieval from millions of documents feasible in milliseconds. Models like SBERT, E5, and text-embedding-ada-002 are bi-encoders trained for semantic similarity tasks. The quality of the bi-encoder is arguably the most important factor in RAG pipeline performance, as it determines which documents the LLM gets to see.