Qdrant 1.20 adds BM25 scoring on full-text indexes and a code tokenizer

BM25 scoring can now be applied to a full-text index. Previously, BM25 required converting text into sparse vectors through an inference step, and the average document length had to be tracked manually. Qdrant now derives the BM25 statistics from the indexed data, and k and b can be set per query. Queries use the new text query type with the indexed field named in the using parameter.

A new code tokenizer splits identifiers on underscores, case changes and acronyms, so getUserName, get_user_name and user name match each other. A new nested parameter on payload indexes creates per-element indexes that can answer nested filters without reading payloads. In Qdrant's benchmarks on 100,000 points, these indexes were 7 to 38 times faster than regular indexes with payload reads.

Two TurboQuant datatypes are added: turbo16, which halves float32 storage with no accuracy loss in the benchmarks, and turbo8, which takes regular float vectors and quantizes them to 8 bits. Memory tiers now also cover the ID tracker, and the internal cluster API requires authentication when an API key is set.

Agent builders can rank keyword results over text already stored in payloads without an embedding step for each document and query.

Clusters that set only a read-only API key will stop communicating between peers after upgrading unless api_key is set or enforce_internal_auth is false.

Collections using inline storage may need more RAM, because original vectors now follow the memory tier of the HNSW index.