PostgreSQL search GIN indexes without a pending list #43

Merged

jcoffey-dev merged 1 commits from fix/pg-gin-fastupdate into main

2026-09-25 00:45:09 +00:00

jcoffey-dev commented

2026-09-24 22:51:30 +00:00

Owner

Problem

Cluster rehearsal 3 on PostgreSQL: searches took about 185 ms with 80 to 260 pending pages in the full-text GIN indexes. They dropped to 2 to 6 ms right after gin_clean_pending_list() or VACUUM, then crept back up as mail arrived.

The search tables' GIN indexes (crates/store/src/backend/postgres/main.rs:269) were created with the default fastupdate=on. New entries wait in an unindexed pending list, and every search scans that list in full. The list is merged only by VACUUM or when it passes 4 MB. Autovacuum only visits an insert-only table after thousands of inserts (about 7k here).

Change

New indexes. The search GIN indexes are created WITH (fastupdate = off), so an insert updates the index at once.
Existing databases. The schema step already runs at every startup (SearchStore::create_indexes → create_search_tables), so the switch happens there. For each GIN index whose reloptions don't already turn fastupdate off:
- ALTER INDEX … SET (fastupdate = off). This takes a SHARE UPDATE EXCLUSIVE lock, which blocks neither reads nor writes.
- Then one gin_clean_pending_list() merges the backlog.
Cost after that. Once an index has been switched, a startup costs one catalog read per index.
Failures. If the switch fails, the error is logged and startup goes on. Search still works, just slower.

Autovacuum per-table settings: skipped. The pending list was the only reason the insert threshold affected search latency. The defaults already handle dead tuples and freezing, and per-table settings would override any tuning the DBA has done.

MySQL: nothing to change. InnoDB FULLTEXT keeps new entries in an in-memory cache (innodb_ft_cache_size) that queries read directly. It has no deferred on-disk list like GIN's and no setting like fastupdate. Deleted rows are filtered until OPTIMIZE TABLE, but that grows with deletes, not inserts.

Benchmark

Local PostgreSQL 16 test container. The table holds 30,000 email-sized tsvector rows (about 150 lexemes each), vacuumed. Then 3,000 new rows arrive and the query is a two-term @@ search. Script: gin-bench2.sql in the work folder.

	pending pages	search
fastupdate=on, after the 3,000 inserts	425	172–246 ms
after `gin_clean_pending_list()`	0	0.07–0.33 ms
fastupdate=off, 3,000 more inserts	0	0.15–0.24 ms

Inserting the 3,000 rows took 4.03 s with fastupdate on and 4.31 s with it off (+7%). Building the tsvector dominates that time.

Tests

store::search_gin::postgres_gin_fastupdate (new, PostgreSQL): pass. It builds the search schema in a schema of its own and reads pg_class.reloptions:
- A fresh schema has fastupdate=off on every GIN index.
- Next, the option is reset to the default and 500 rows are left pending. One startup then turns fastupdate off on every index and leaves 0 pending tuples (pgstatginindex).
- A second startup changes nothing.
On main the test fails at the first check (fresh index gin_s_cal_atnd has options "").
Command: STORE=PostgreSql cargo test -p tests --features postgres store::search_gin
store::search_tests on PostgreSQL wasn't rerun. It already fails on main (an ordering assertion in query.rs:679) and this change doesn't touch it.

## Problem Cluster rehearsal 3 on PostgreSQL: searches took about 185 ms with 80 to 260 pending pages in the full-text GIN indexes. They dropped to 2 to 6 ms right after `gin_clean_pending_list()` or VACUUM, then crept back up as mail arrived. The search tables' GIN indexes (`crates/store/src/backend/postgres/main.rs:269`) were created with the default `fastupdate=on`. New entries wait in an unindexed pending list, and every search scans that list in full. The list is merged only by VACUUM or when it passes 4 MB. Autovacuum only visits an insert-only table after thousands of inserts (about 7k here). ## Change - **New indexes.** The search GIN indexes are created `WITH (fastupdate = off)`, so an insert updates the index at once. - **Existing databases.** The schema step already runs at every startup (`SearchStore::create_indexes` → `create_search_tables`), so the switch happens there. For each GIN index whose `reloptions` don't already turn fastupdate off: - `ALTER INDEX … SET (fastupdate = off)`. This takes a SHARE UPDATE EXCLUSIVE lock, which blocks neither reads nor writes. - Then one `gin_clean_pending_list()` merges the backlog. - **Cost after that.** Once an index has been switched, a startup costs one catalog read per index. - **Failures.** If the switch fails, the error is logged and startup goes on. Search still works, just slower. **Autovacuum per-table settings: skipped.** The pending list was the only reason the insert threshold affected search latency. The defaults already handle dead tuples and freezing, and per-table settings would override any tuning the DBA has done. **MySQL: nothing to change.** InnoDB FULLTEXT keeps new entries in an in-memory cache (`innodb_ft_cache_size`) that queries read directly. It has no deferred on-disk list like GIN's and no setting like fastupdate. Deleted rows are filtered until `OPTIMIZE TABLE`, but that grows with deletes, not inserts. ## Benchmark Local PostgreSQL 16 test container. The table holds 30,000 email-sized `tsvector` rows (about 150 lexemes each), vacuumed. Then 3,000 new rows arrive and the query is a two-term `@@` search. Script: `gin-bench2.sql` in the work folder. | | pending pages | search | |---|---|---| | fastupdate=on, after the 3,000 inserts | 425 | 172–246 ms | | after `gin_clean_pending_list()` | 0 | 0.07–0.33 ms | | fastupdate=off, 3,000 more inserts | 0 | 0.15–0.24 ms | Inserting the 3,000 rows took 4.03 s with fastupdate on and 4.31 s with it off (+7%). Building the `tsvector` dominates that time. ## Tests - `store::search_gin::postgres_gin_fastupdate` (new, PostgreSQL): **pass**. It builds the search schema in a schema of its own and reads `pg_class.reloptions`: - A fresh schema has `fastupdate=off` on every GIN index. - Next, the option is reset to the default and 500 rows are left pending. One startup then turns fastupdate off on every index and leaves 0 pending tuples (`pgstatginindex`). - A second startup changes nothing. - On `main` the test fails at the first check (`fresh index gin_s_cal_atnd has options ""`). - Command: `STORE=PostgreSql cargo test -p tests --features postgres store::search_gin` - `store::search_tests` on PostgreSQL wasn't rerun. It already fails on `main` (an ordering assertion in `query.rs:679`) and this change doesn't touch it.

jcoffey-dev added 1 commit 2026-09-24 22:51:31 +00:00

PostgreSQL search GIN indexes without a pending list

ci / fork-checks (pull_request) Successful in 30s

Details

ci / build (pull_request) Successful in 7m26s

Details

fde43774b4

A three-node rehearsal on PostgreSQL saw searches take about 185 ms
with 80 to 260 pages in the full-text indexes' pending lists, 2 to 6 ms
right after gin_clean_pending_list() or VACUUM, then creep back up as
mail came in. The search tables' GIN indexes were created with the
default fastupdate=on: new entries wait in an unindexed pending list
that every search scans in full until VACUUM (or 4 MB of backlog)
merges it, and autovacuum only visits an insert-only table after
thousands of inserts.

The search GIN indexes are now created WITH (fastupdate = off), so an
insert pays its index update at once. The schema step runs at every
startup (create_search_tables, via SearchStore::create_indexes), so
indexes made before this change are switched there: when an index's
reloptions don't already turn fastupdate off, ALTER INDEX ... SET
(fastupdate = off) and one gin_clean_pending_list() merge its backlog.
The ALTER takes a SHARE UPDATE EXCLUSIVE lock, which blocks neither
reads nor writes; after the first startup the step is one catalog read
per index. A failure is logged and startup goes on (search still
works, only slower).

Per-table autovacuum settings for the search tables are left alone.
The pending list was the only reason the insert threshold mattered for
search; dead tuples and freezing are served by the defaults, and table
settings would override whatever tuning the DBA has done.

MySQL is unaffected: InnoDB FULLTEXT keeps new entries in an in-memory
cache that queries read directly, with no setting like fastupdate.

store::search_gin::postgres_gin_fastupdate (new, PostgreSQL) builds
the search schema in a schema of its own and checks pg_class.reloptions:
fastupdate=off on every GIN index of a fresh schema; then, with the
option reset to the default and 500 rows pending, one startup turns it
off everywhere and leaves no pending tuples (pgstatginindex); a second
startup changes nothing. On main it fails at the first check.

jcoffey-dev merged commit 181ab1c140 into main

2026-09-25 00:45:09 +00:00

jcoffey-dev deleted branch fix/pg-gin-fastupdate

2026-09-25 00:45:09 +00:00

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: inbuxa/inbuxa-server#43