mrkeyoor.com_
Fri 16 Oct 18:52 UTC
Open Source6 min read

pg_vectorsearch PRISM Can Return Too Few Results After Filtering

Tiger Data reports lower memory use with its new PostgreSQL vector index. The alpha can still return too few filtered results, and growing datasets need manual maintenance.

Ask pg_vectorsearch's new PRISM index for ten nearest matches, add a filter, and you can get fewer than ten results even when more matching rows exist. The 0.1.0 release notes document that limitation. For a developer evaluating the newly released PostgreSQL extension, it's a reason to test the application's actual queries before treating a fast similarity benchmark as a migration decision.

Tiger Data announced pg_vectorsearch on October 13, with a partition-based index aimed at reducing the memory needed for vector search. The repository labels it alpha software for testing and warns that prerelease upgrades may require dropping and recreating the extension and its indexes. There's working code to evaluate, with explicit limits on what it can replace today.

Smaller machines, compared at the same recall

In Tiger Data's tests, PRISM indexed 100 million Cohere embeddings on a machine with 128 GiB of RAM in one hour and 40 minutes. At 95% recall@10, its median query latency was 6 milliseconds. Pgvector's HNSW needed the 512 GiB machine to finish building, taking 42 hours, and returned 8-millisecond median latency at that recall. The reported index sizes were 25 GB and 180 GB, respectively.

Those are vendor measurements. The comparison used pgvector's half-precision halfvec representation, and the company says both indexes used the same PostgreSQL configuration. Its public Rekall benchmark tool lets others inspect and rerun comparisons. Recall measures how many of the true nearest neighbours the approximate search finds. Matching recall matters: a faster query that misses more neighbours may be a poor trade for your application.

How PRISM searches a smaller index

The architecture document describes a tree of cluster centres that routes a query toward promising groups of vectors. Within those groups, PRISM scans compact entries stored together in posting lists. Frequently accessed routing pages use PostgreSQL's existing shared buffer cache. The design avoids needing a separate search service or a dedicated cache just for those pages.

PRISM uses RaBitQ quantization, a compressed representation with error bounds, to reject candidates before fetching their full vectors for a more expensive comparison. Surviving candidates are reranked against those full vectors. The documented read pattern is sequential within each posting list, with random access still needed between lists and when fetching vectors for reranking. Compression reduces the data scanned, while the final comparison still depends on table access.

That document also mixes implemented features with designs. Its multi-tenant section explicitly says composite-key indexes aren't implemented, despite describing their intended operation. A team searching a shared table by customer ID needs to distinguish those plans from the single-column index available now.

Existing embeddings can stay where they are

The README describes compatibility with existing pgvector vector and halfvec columns. Installing pg_vectorsearch alongside pgvector lets you build PRISM on the existing column without rewriting or copying it. The extension also supplies its own vec32 and vec16 types for new tables that don't use pgvector. That lowers the work needed to compare indexes on a test database.

For an already populated test table named documents with a pgvector vector column, the documented L2-distance setup is short:

CREATE EXTENSION pg_vectorsearch;
CREATE INDEX documents_embedding_prism
    ON documents USING prism (embedding vec32_l2_ops);

Check the query plan before comparing timings. The README's example uses EXPLAIN to verify that PostgreSQL chooses PRISM when both indexes exist. It also requires PostgreSQL 18 or later. Compatibility with your embedding column doesn't remove that server-version requirement, and unchanged SQL doesn't establish unchanged retrieval quality.

A filter changes the search problem

PRISM's tuning guide explains that the scan calculates a candidate pool before PostgreSQL applies the query's filter. It enlarges that pool using the estimated selectivity of the WHERE condition. If matching rows aren't distributed as the estimate expects, too few can survive. The pool is also bounded by work_mem, so asking for a large result set can run into a separate memory limit.

The release notes say the scan doesn't go back for more candidates when filtered results come up short. They also warn that an ordered scan asking for more rows than its buffer can hold is silently truncated. For an application, this means testing returned row counts alongside latency. A query that completes quickly with an incomplete page of results needs a different evaluation from an unfiltered nearest-neighbour query.

Pgvector documents the same general difficulty with filtering after approximate search, but its iterative scans, available since version 0.8.0, can keep scanning until enough results are found or a configured limit is reached. Its documentation also discusses indexes on filter columns, partial indexes and partitioning. These are concrete differences to include when comparing PRISM with an existing HNSW deployment.

Customer-scoped retrieval makes the distinction easy to see. A shared index may find nearby vectors belonging to other customers, which the filter then removes. Pgvector's multitenancy guidance says vectors from one tenant can affect another tenant's recall and speed in a shared approximate index, and recommends separate tables or list partitioning for isolation. Test the structure you actually use, including customers with relatively few documents.

Inserts work, but partitions need attention

New rows append to their assigned posting lists, which don't split automatically, according to the maintenance guide. Rebuilding with REINDEX adjusts the index to the dataset's new size. Manual splitting is available, but it currently requires a flat, single-level index with RaBitQ centroid pages. The guide says the default build doesn't meet those conditions.

That restriction matters when reading the release's reference to prism_rebalance. The detailed maintenance instructions say unsupported index shapes raise an error. Rebalancing also leaves replaced posting lists on disk, so splitting can increase disk use until a rebuild reclaims the space. A maintenance experiment should record the index shape and disk growth, rather than assume every PRISM index can use the procedure.

Deletes have another storage consequence. The changelog says VACUUM marks deleted entries so scans can skip them, but their index space isn't reused yet. It calls for REINDEX to reclaim that space. A largely static document collection and a frequently replaced collection therefore deserve separate tests, even if their initial search timings look similar.

Compare the queries your users send

The main query-time control is prism.nprobe, the number of nearby clusters searched. The tuning guide says its automatic default targets roughly 95% recall@10. Raising it searches more clusters at a latency cost. Treat that target as a starting point to measure on your data, especially when a missed document changes the answer your application produces.

Rekall measures recall against throughput across PRISM, pgvector indexes and VectorChord. It records PostgreSQL settings and extension versions, and supports concurrent-client tests as well as separate loading, index-building and search phases. Those records make a result easier to check than a latency number without its configuration. MrKeyoor hasn't independently reproduced Tiger Data's results.

For an application trial, the filtered-scan limitation supplies a useful acceptance question: does each representative query return enough eligible documents? Compare those results with exact search, then repeat after inserts and replacements. Include selective filters that leave very few eligible rows. That checks the behaviour behind a search page or retrieval step, beyond whether the index wins a timed query.

The next release to watch is one that addresses scan continuation and index upkeep. Tiger Data's stated work toward general availability includes streaming filtered search and automatic rebalancing. Until those changes ship and are tested, keep the filtered queries and growing-data cases in your comparison suite. They're where this alpha's documented limits can become visible to users.

We reviewed this

  1. Search — our honest review
  2. query — our honest review
  3. vector — our honest review

Sources

  1. pg_vectorsearch repository and README
  2. pg_vectorsearch 0.1.0 changelog
  3. Announcing pg_vectorsearch: Scalable Open-Source Vector Search in PostgreSQL
  4. PRISM architecture
  5. PRISM tuning guide
  6. PRISM index maintenance
  7. pgvector filtering, iterative scans and multitenancy
  8. Rekall vector-search benchmark tool