UC Berkeley's LEANN open-source vector index cuts RAG index storage sharply

UC Berkeley's LEANN is an open-source vector index that keeps a pruned graph instead of full embeddings and recomputes embeddings when a query runs. MarkTechPost reports that it shrinks a 60-million-chunk RAG index from 201 GB to 6 GB.

In the comparison given in the article, LEANN matches HNSW accuracy at 90% recall using 4 GB where HNSW uses 188 GB. The trade-off is search latency, which the article describes as seconds-level.

Teams running large retrieval indexes for agents may be able to cut storage costs considerably with this approach.

Each query recomputes embeddings, so latency-sensitive agents will need to test whether seconds-level search is acceptable. Figures come from the article, so benchmark on your own corpus before switching indexes.