Locating Known Objects Using Two Cameras

Abstract

Locating objects is a common problem in geometric computer vision. The goal is to determine the coordinates of an object in a scene. When the object is projected into the camera plane, the depth information is lost, and to overcome this, two cameras are used. Several approaches can be found in the literature on how to reconstruct a scene with two cameras. Binocular stereo vision tries to emulate the human vision by placing the two cameras next to each other, and other methods, such as the one studied here, use cameras far away from each other and in a steep angle to each other.

This thesis investigates the latter cae. Due to the large difference in what the cameras can see, a full reconstruction of the scene would be difficult. The method used in this study uses a fully calibrated approach where two cameras are calibrated both individually and relative to each other. With the known relative positions of the cameras in addition to the internal parameters of both cameras, points of interest on both images can be triangulated into the 3D space. The points are selected based on the corners of the bounding boxes of the objects, which are detected using state-of-the-art Mask R-CNN object class and segment detector. With this method, a bounding box is determined in 3D.

Finally, the empirical results of the method and the effects of long separation of the cameras are presented. According to the evaluation, the 3D localization accuracy improves with camera separation. The evaluation shows that the calibration method also affects the accuracy of the localization and with errors in calibration the final bounding box will not be consistent.