Empirical Power-performance Analysis of Layer-wise CNN Inference on Single Board Computers

This paper analyzes the impact of input sparsity and DFS/DVFS configurations for single-board computers on the execution time, power, and energy of each VGG16 layer as the first step towards efficient CNN inference on single-board computers. For this purpose, we first develop a power and execution time measurement environment and perform experiments using Raspberry Pi 4 and NVIDIA Jetson Nano. Our results show that clock frequency strongly correlates with execution time and power. Inversely, input sparsity has a weak correlation with execution time and power. Then, we show that a coarse-grained DVFS model can explain over 96% of the variations in the power of each VGG16 layer even when sets of clock frequency and voltage on the single-board computer are unavailable.