Geek Out Time: When Geometry Fights Back — Why Your Embeddings in RAG Can’t Think in ‘AND’

In information retrieval, dense embeddings have become the standard way to represent and retrieve text. With better models and larger dimensions, we expect better recall.

But a new paper from Google DeepMind — “On the Theoretical Limitations of Embedding-Based Retrieval” (Aug 2025) ( https://arxiv.org/abs/2508.21038 ) — shows something counterintuitive: no matter how large or well-trained your embedding model is, there are combinations of relevant documents it can never represent. Interesting ha?

Even simple queries can exceed the geometric capacity of the embedding space. I decided to replicate this in a small, controlled experiment.

The Task

This is an excerpt — the full article continues on Medium.