Taking a cue from Prometheus, we invert typical replication from push to pull.
Concretely, all ingest and storage nodes join a cluster, and gossip knowledge of each other.
All storage nodes consume segments regularly and randomly from all ingest nodes.
Consumed segments are merged and, after they reach a certain age or size, replicated across other storage nodes.
Only once they are successfully replicated are the original segments confirmed and cleared from the ingest nodes.
Group items matching
in title, tags, annotations or urlPeter Bourgon · OK Log - 1 views
E.W.Dijkstra Archive - 0 views
1 - 3 of 3
Showing 20▼ items per page