Skyplane: Optimizing transfer cost and throughput using cloud-aware overlays

Meta Info

Presented in NSDI 2023.

Authors: Paras Jain, Sam Kumar, Sarah Wooders, Shishir G Patil, Joseph E Gonzalez, Ion Stoica, University of California, Berkeley

Homepage: https://skyplane.org/

Code: https://github.com/skyplane-project/skyplane

Understanding the paper

TL;DRs

This paper presents Skyplane, a system for bulk data transfer between cloud object stores that uses cloud-aware network overlays to optimally navigate the trade-off between price and performance.

Its planner uses mixed-integer linear programming to determine the optimal overlay path and resource allocation for data transfer, subject to user-provided constraints on price or performance.

Comparison to Existing Work

• Consider price and elasticity (opposed to Resilient overlay networks (RON), SOSP 2001).

• Richer problem and solution space.

Implementation

• Implemented in Python 3.

• Use the Gurobi library to solve MILP instances.

• Support three major cloud providers: Amazon Web Services, Microsoft Azure, and Google Cloud Platform.