Scale-out storage: sharded blob, in-memory and lookup stores; configured read replicas reported (ST-1 to ST-4, ST-16 to ST-30)

A Sharded blob store places each blob on xxh3(key) mod N, over the whole key; reads fall back to the other members, so blobs placed under an earlier member list stay readable, and deletes find them wherever they are. The member list is recorded in the data store (secrets left out): added or reordered members are a warning, a missing one refuses to open. Blobs are compressed and marked before they reach a member. A Sharded in-memory or lookup store sends each key to its home Redis member, and prefix deletes and purges to all; a node whose member list differs from the recorded one logs an error and runs on. Members are checked for duplicates and must all open. Until read-replica routing is built, each configured replica is reported at startup instead of being silently ignored, and nothing connects to it. The new scaleout_blob_tests covers tests 2 to 7, and the existing blob suite passes against three FileSystem members (BLOB_STORE=Sharded).
2026-09-19 10:11:30 -07:00
parent e7efa91cbc
commit e5e326de6b
16 changed files with 1144 additions and 84 deletions
@@ -0,0 +1,68 @@
+/*
+ * SPDX-FileCopyrightText: 2026 Coffey Labs
+ *
+ * SPDX-License-Identifier: AGPL-3.0-only
+ */
+
+//! Scale-out storage (`docs/spec/features/scale-out-storage.md`): sharded
+//! blob stores (ST-16 to ST-22) and sharded in-memory and lookup stores
+//! (ST-23 to ST-29). Each is one more variant of the store enums, whose
+//! members are ordinary stores.
+
+pub mod blob;
+pub mod layout;
+pub mod memory;
+
+pub use blob::ShardedBlobStore;
+pub use memory::ShardedInMemoryStore;
+
+/// A key's home: `xxh3_64(key) mod N`, seed 0, over the whole key (ST-16).
+/// Fixed forever once shipped.
+pub fn home(key: &[u8], members: usize) -> usize {
+    (xxhash_rust::xxh3::xxh3_64(key) % members.max(1) as u64) as usize
+}
+
+/// A URL without its user information, for records and logs.
+pub fn without_credentials(url: &str) -> String {
+    match url.split_once("://") {
+        Some((scheme, rest)) => {
+            let rest = match rest.split_once('/') {
+                Some((authority, path)) => {
+                    let host = authority.rsplit_once('@').map_or(authority, |(_, h)| h);
+                    format!("{host}/{path}")
+                }
+                None => rest.rsplit_once('@').map_or(rest, |(_, h)| h).to_string(),
+            };
+            format!("{scheme}://{rest}")
+        }
+        None => url.rsplit_once('@').map_or(url, |(_, h)| h).to_string(),
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn places_and_hides_credentials() {
+        // Stable placement: these values must never change (ST-16)
+        assert_eq!(home(b"", 3), (xxhash_rust::xxh3::xxh3_64(b"") % 3) as usize);
+        let spread = (0u32..3000)
+            .map(|n| home(&n.to_be_bytes(), 3))
+            .fold([0; 3], |mut acc, h| {
+                acc[h] += 1;
+                acc
+            });
+        assert!(spread.iter().all(|n| *n > 800), "{spread:?}");
+
+        assert_eq!(
+            without_credentials("redis://user:secret@host:6379/0"),
+            "redis://host:6379/0"
+        );
+        assert_eq!(without_credentials("rediss://:pw@host"), "rediss://host");
+        assert_eq!(
+            without_credentials("redis://host:6379"),
+            "redis://host:6379"
+        );
+    }
+}