Anole: Adapting Diverse Compressed Models for Cross-scene Prediction on Mobile Devices

Abstract

Emerging Artificial Intelligence of Things (AIoT)applications desire online prediction using deep neural network(DNN) models on mobile devices. However, due to the movementof devices,unfamiliartest samples constantly appear, significantlyaffecting the prediction accuracy of a pre-trained DNN. Inaddition, unstable network connection calls for local modelinference. In this paper, we propose a light-weight scheme, calledAnole, to cope with the local DNN model inference on mobiledevices. The core idea of Anole is to first establish an armyof compact DNN models, and then adaptively select the modelfitting the current test sample best for online inference. The keyis to automatically identifymodel-friendlyscenes for trainingscene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combiningboth human heuristics and feature similarity in separating scenes.Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implementAnole on different types of mobile devices and conduct extensivetrace-driven and real-world experiments based on unmannedaerial vehicles (UAVs). The results demonstrate that Anoleoutwits the method of using a versatile large DNN in terms ofprediction accuracy (4.5% higher), response time (33.1% faster)and power consumption (45.1% lower).