SE-Radio Episode 285: James Cowling on Dropbox’s Distributed Storage System

Hardware Logistics

James Cowling discusses the logistical hurdles Dropbox faced during their transition from Amazon's S3 to their own distributed storage system. The process involved acquiring and implementing large volumes of hardware, which required a robust supply chain management system to handle unexpected events like truck crashes that resulted in hardware loss 1. Cowling highlights the importance of having a multidisciplinary team to manage these challenges, stating, "We were adding maybe five plus petabytes a day, and we had two trucks crash in one week" 1. This experience underscored the need for a mature supply chain organization to ensure continuous operation.

   

Network Management

Managing network connectivity was crucial for Dropbox during their massive data migration. James Cowling explains that controlling the network allowed Dropbox to optimize data processing and storage, enhancing user experience by co-locating services with storage 1. He notes, "We have three to five times lower latency across our percentiles than we did in an S3," emphasizing the technical and cost advantages of their tailored storage system 2. This strategic network management facilitated the efficient transfer of over 500 petabytes of data from Amazon to Dropbox's system.

   

Architectural Decisions

Architectural decisions were pivotal in accommodating Dropbox's scaling storage demands. James Cowling highlights the importance of maintaining simple, clean APIs to manage complexity and avoid layer violations, which can hinder performance 3. He recounts a critical moment when a bug reached the staging cluster, prompting a reset of the production validation clock to ensure system reliability 4. Cowling emphasizes the team's commitment to launching a system they were "unconditionally confident in," even if it meant delaying the launch to address unforeseen issues.