Scale-out storage: PostgreSQL and MySQL read replicas (ST-5 to ST-15)
A data store with readReplicas becomes a replicated store. Writes, operator-written SQL and everything outside a read scope go to the primary. JMAP reads before a request's first write, IMAP LIST, STATUS, SEARCH, SORT and FETCH, POP3 RETR and TOP, DAV GET, PROPFIND and REPORT, and blob downloads run in a read scope. Only account data (properties, indexes, change logs, counters, ACLs, blobs, the search index) is read from a replica; the registry, in-memory values, the task queue and the rest stay on the primary. In a scope, the first read picks a replica round-robin among those up and under the lag limit, and only if it has every change this node has written or heard of for the scope's accounts: marks come from write results, the cluster's state-change broadcasts, a sinceState the client presents, and, with more than one node, Redis. A write inside the scope sends the rest of it to the primary. A miss on a replica is looked up on the primary, and a replica error retries the read there and marks the replica down. Each node samples lag every second (WAL positions on PostgreSQL; GTID sets or Seconds_Behind_Source on MySQL), stops reading from a replica over 5 s and starts again under 2.5 s, and probes a down replica every 10 s. At startup a replica is left out if it's the primary, isn't read-only, applies out of commit order, or doesn't show a marker written to the primary within six tries. replica_tests (postgres, STORE=PostgreSqlReplicated) runs a primary and a streaming hot standby in containers: tests 9, 10, 12, 13, 14 and 15 pass.
This commit is contained in:
@@ -151,7 +151,13 @@ impl ParseHttp for Server {
|
||||
path.next().and_then(BlobId::from_base32),
|
||||
path.next(),
|
||||
) {
|
||||
return match self.blob_download(&blob_id, &access_token).await? {
|
||||
// inbuxa: ST-6: a download may be served by a read replica
|
||||
let blob = store::backend::scaleout::replica::replica_read(
|
||||
access_token.all_ids().map(|account_id| (account_id, 0)),
|
||||
self.blob_download(&blob_id, &access_token),
|
||||
)
|
||||
.await?;
|
||||
return match blob {
|
||||
Some(blob) => Ok(DownloadResponse {
|
||||
filename: name.to_string(),
|
||||
content_type: req
|
||||
|
||||
Reference in New Issue
Block a user