Vehicle Pose and Shape Estimation in UAV Imagery Using a CNN

Abstract

Vehicle reconstruction from single aerial images is an important but challenging task. In this work, we introduce a new framework based on convolutional neural networks (CNN) that performs monocular detection, pose, shape and type estimation for vehicles in UAV imagery, taking advantage of a strong 3D object model. In the final training phase, all components of the model are trained end-to-end. We present a UAV-based dataset for the evaluation of our model and additionally evaluate it on an augmented version of the Hessingheim benchmark dataset. Our method presents encouraging pose and shape estimation results: Based on images of 3 cm GSD, it achieves median errors of up to 5 cm in position and 3◦ in orientation, and RMS errors of ±7 cm and ±24 cm in planimetry and height, respectively, for keypoints describing the car shape.