Serving unseen deep learning model with near-optimal configurations: A fast adaptive search approach

Meta Info

Presented in SoCC 2022.

Authors: Yuewen Wu, Heng Wu, Diaohan Luo, Yuanjia Xu, Yi Hu, Wenbo Zhang, Hua Zhong, Institute of Software, Chinese Academy of Sciences

Code: https://github.com/dos-lab/Falcon

Understanding the paper

TL;DRs

This paper presents Falcon, a novel configuration recommender system that can quickly adapt to unseen DL models.

Existing Problem

There exists a cold start problem when searching a configuration of DL models.

Key Insights

• Some Key Operators (KOPs) can be used to estimate the performance of DL models.

• The resource sensitivity can be represented by four typical Key Operator Resource Curves (KOP-RCs), including Slope, Convex, Concave, and Plane.

Solutions

• A DL model can be characterized by its KOPs and corresponding KOP-RCs.

• Use a combination of Monte Carlo Tree Search and Bayesian Optimization (MCTS-BO) to search the configurations.

Evaluation

• Compared to Morphling.