SE Radio 560: Sugu Sougoumarane on Distributed SQL Databases

Raft vs Paxos

Sugu Sougoumarane explains the choice of Raft over Paxos for Vitess, emphasizing practicality and failure detection. Raft's ability to include failure detection makes it more suitable for real-world applications compared to Paxos, which lacks this feature. Sugu highlights a modified quorum definition used at YouTube, allowing for flexible deployments with numerous replicas, ensuring data durability even under high query loads.

The one improvement that we made over raft is there is actually a paper called flex pack source, which actually makes a modification in how you select your quorums, which gives you a huge flexibility in terms of your deployments.

--- Sugu Sougoumarane

This flexibility has proven effective, with no data loss reported by major users like YouTube and Slack 1.

   

Durability

Vitess provides robust durability guarantees through its consensus system integration, offering customizable configurations for reliability and safety. Sugu describes Vitess's pluggable durability policy, allowing users to specify durability levels across zones or regions, adapting to complex cloud architectures. This approach ensures data integrity and availability, even during regional failures, by maintaining operations in a diminished capacity if necessary.

Vitess actually uses what I believe is a more generalized form of consensus, which actually allows you to come up with more practically useful topologies.

--- Sugu Sougoumarane

Such flexibility has enabled Vitess to meet the demands of high-traffic platforms like JD and Slack without data loss 1 2.

   

Automation

Vitess's system automation focuses on self-healing capabilities, minimizing human intervention during node failures. Sugu outlines the importance of distributed durability, availability, and automation, ensuring that systems continue to function despite node failures. By incorporating time components, Vitess's consensus protocols allow for efficient failover processes, distinguishing it from traditional Paxos systems.

The system has to be able to heal itself without data loss.

--- Sugu Sougoumarane

This automation reduces the need for manual oversight, enhancing reliability and efficiency in maintaining data integrity 3.