4 Commits

Include renames

Author	SHA1	Message	Date
jcoffey-dev	6e50ba25a9	SQL pools time out; task locks are a renewed five-minute lease ... A 3-node rehearsal (PostgreSQL + NATS + Garage) found two ways a crash leaves work stuck: Pool hangs. The PostgreSQL pool (deadpool) was built with no timeouts, so a request waited for a free connection, and for one to be opened or recycled, for as long as it took: forever when the server stopped answering. MySQL's pool (mysql_async) has no wait timeout at all. - PostgreSQL: wait 30 s (or the store's timeout if longer), create the store's timeout or 15 s (it bounds the whole handshake, where tokio-postgres's connect_timeout covers only the TCP connect), recycle 10 s. The pool config is now always set, not only with poolMaxConnections. - MySQL: every connection is taken through MysqlStore::conn(), which gives up after 30 s. - Both: TCP keepalive after 60 s idle, so a server that vanished without closing the connection is noticed in minutes rather than the two-hour system default. The DataStore schema has no pool timeout settings, so these are fixed defaults; the store's own timeout bounds connecting on PostgreSQL. Task locks. A task lock lasted an hour, so after a hard crash the dead node's tasks waited up to an hour and five minutes. The lock is now a five-minute lease: while this node runs a task, the task manager renews its lock every third of the lifetime (InMemoryStore::renew_lock, a compare-and-set on the store backends and SET XX EX on Redis, which leaves a lock that already expired alone). A killed node's tasks run elsewhere within about five minutes plus the claim recheck. A task this node holds isn't handed to a worker again by the scan. store::pool_timeout (new): a local listener that accepts connections and never answers plays a hung server; a PostgreSQL store with a 2 s timeout returns an error in about 4 s, and a MySQL store in 30 s. Without the timeouts both wait for good. store::task_locks gains a task held for 1.5 lock lifetimes: its lease is still held, and released when the task ends.	2026-09-24 13:11:26 -07:00
jcoffey-dev	6a53d47106	Mark the files this fork changed (AGPL section 5(a)) ... The AGPL asks a modified version to carry prominent notices saying it was modified, and giving a date. Publishing the source is the conveyance that asks for it, so it wants doing before the repository is public rather than at the release. Every upstream file the fork changed now says so in its header, beneath the notice it came with: 164 files, found by diffing against the upstream snapshot branch rather than by guessing, so the list is what actually differs. Files the fork wrote itself already carry their own copyright and need nothing. Upstream's notices are untouched, which its licence requires and which was already true. The README says the same thing in prose, since the obligation is on the work as a whole and not only its Rust files. Builds unchanged: the server and the test binary both compile.	2026-09-19 23:48:35 -07:00
jcoffey-dev	e5e326de6b	Scale-out storage: sharded blob, in-memory and lookup stores; configured read replicas reported (ST-1 to ST-4, ST-16 to ST-30) ... A Sharded blob store places each blob on xxh3(key) mod N, over the whole key; reads fall back to the other members, so blobs placed under an earlier member list stay readable, and deletes find them wherever they are. The member list is recorded in the data store (secrets left out): added or reordered members are a warning, a missing one refuses to open. Blobs are compressed and marked before they reach a member. A Sharded in-memory or lookup store sends each key to its home Redis member, and prefix deletes and purges to all; a node whose member list differs from the recorded one logs an error and runs on. Members are checked for duplicates and must all open. Until read-replica routing is built, each configured replica is reported at startup instead of being silently ignored, and nothing connects to it. The new scaleout_blob_tests covers tests 2 to 7, and the existing blob suite passes against three FileSystem members (BLOB_STORE=Sharded).	2026-09-19 10:11:30 -07:00
jcoffey-dev	7dae9b29fd	Import upstream v0.16.22, stripped ... Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f Enterprise-only files removed or emptied: 63 Enterprise-only snippets removed: 117 in 50 files Dangling module declarations removed: 5 Cargo edits turning enterprise off: 14 Verification: clean Enterprise feature gates left for rebuilt features: 19 in 18 files Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main.	2026-09-18 10:21:56 -07:00