Skip to Main content Skip to Navigation
Conference papers

Execution Time Modeling for CNN Inference on Embedded GPUs

Abstract : Machine learning is one of the most cutting edge methods in computer vision. Convolutional Neural Networks (CNN) in particular are widely used in edge computing based applications such as autonomous driving for image recognition or object tracking. Different constraints exist in this application area such as real-time, energy consumption, memory resources, etc. Choosing the optimal CNN for each GPU at hand is really hard to do, while maintaining high levels of accuracy and performance. This makes prior knowledge about the execution time a necessary prerequisite information before the final deployment of the CNN on the edge GPU platform. In this paper, we compare 5 execution time prediction models on a large set of CNNs-based applications. The tested predictors use machine learning regression approach. The proposed methodology is based on the utilization of high level CNN features. At the opposite of state-of-the-art approaches, no implementation or profiling on the hardware is required. A Mean Absolute Percentage Error (MAPE) of 5% using Support Vector Regression and Artificial Neural Networks has been obtained in the experiments. Our comparison shows the efficiency of these models to rapidly explore a large space of CNN models or Hardware configurations.
Document type :
Conference papers
Complete list of metadata
Contributor : Kathleen Torck Connect in order to contact the contributor
Submitted on : Monday, October 18, 2021 - 8:44:47 AM
Last modification on : Wednesday, October 27, 2021 - 1:16:02 PM




Noureddine Bouhali, Hamza Ouarnoughi, Smail Niar, Abdessamad Ait El Cadi. Execution Time Modeling for CNN Inference on Embedded GPUs. DroneSE and RAPIDO '21: Methods and Tools, Jan 2021, Budapest, Hungary. pp.59-65, ⟨10.1145/3444950.3447284⟩. ⟨hal-03381837⟩



Record views