PostgreSQL search GIN indexes without a pending list
A three-node rehearsal on PostgreSQL saw searches take about 185 ms with 80 to 260 pages in the full-text indexes' pending lists, 2 to 6 ms right after gin_clean_pending_list() or VACUUM, then creep back up as mail came in. The search tables' GIN indexes were created with the default fastupdate=on: new entries wait in an unindexed pending list that every search scans in full until VACUUM (or 4 MB of backlog) merges it, and autovacuum only visits an insert-only table after thousands of inserts. The search GIN indexes are now created WITH (fastupdate = off), so an insert pays its index update at once. The schema step runs at every startup (create_search_tables, via SearchStore::create_indexes), so indexes made before this change are switched there: when an index's reloptions don't already turn fastupdate off, ALTER INDEX ... SET (fastupdate = off) and one gin_clean_pending_list() merge its backlog. The ALTER takes a SHARE UPDATE EXCLUSIVE lock, which blocks neither reads nor writes; after the first startup the step is one catalog read per index. A failure is logged and startup goes on (search still works, only slower). Per-table autovacuum settings for the search tables are left alone. The pending list was the only reason the insert threshold mattered for search; dead tuples and freezing are served by the defaults, and table settings would override whatever tuning the DBA has done. MySQL is unaffected: InnoDB FULLTEXT keeps new entries in an in-memory cache that queries read directly, with no setting like fastupdate. store::search_gin::postgres_gin_fastupdate (new, PostgreSQL) builds the search schema in a schema of its own and checks pg_class.reloptions: fastupdate=off on every GIN index of a fresh schema; then, with the option reset to the default and 500 rows pending, one startup turns it off everywhere and leaves no pending tuples (pgstatginindex); a second startup changes nothing. On main it fails at the first check.
This commit is contained in:
@@ -21,6 +21,8 @@ pub mod replica_mysql; // inbuxa: read replicas on MySQL
|
||||
#[cfg(all(feature = "postgres", feature = "redis"))]
|
||||
pub mod replica_cluster; // inbuxa: read replicas across nodes
|
||||
pub mod scaleout; // inbuxa: scale-out storage
|
||||
#[cfg(feature = "postgres")]
|
||||
pub mod search_gin; // inbuxa: GIN indexes without a pending list
|
||||
#[cfg(any(feature = "postgres", feature = "mysql"))]
|
||||
pub mod sql_timeout;
|
||||
pub mod task_locks; // inbuxa: task locks across nodes
|
||||
|
||||
Reference in New Issue
Block a user