3D tracking of a mobile device
Tuokko, Miro-Markus (2020)
2020
Automaatiotekniikan DI-tutkinto-ohjelma - Degree Programme in Automation Engineering, MSc (Tech)
Tekniikan ja luonnontieteiden tiedekunta - Faculty of Engineering and Natural Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2020-09-10
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202008296758
https://urn.fi/URN:NBN:fi:tuni-202008296758
Tiivistelmä
There are multiple different methods for tracking the location and orientation of a rigid object. Finnish company Piceasoft Ltd. has a need to track a mobile phone while it is being turned by hand. The tracker needs to be running on an Android phone. In this thesis, different tracking methods are studied. How well the methods would fit this application is evaluated. The edgebased tracking method is chosen as the most fitting because it does not require a training stage, and the same tracker can be used for tracking different mobile phones. The edge-based method is also light enough so it can be executed on a mobile device.
Mathematical concepts that are needed in the implementation of an edge-based tracker are discussed in detail. The first step in the tracking process is to project a 3D model of the object to the image plane. Internal and external matrices are used in perspective projection. The outermost edges of the projected model are found next because the inner edges are not used by the tracker. The outermost edges are divided into control points.
The image from the camera is used to find the corresponding points for the control points. A search line is drawn for each one of the control points. The search lines are perpendicular to the outermost edges of the model. A measurement point should be located on each of the search lines. The measurement points are located on the edges of the phone that is in the image. The search lines are rotated and stacked on top of each other to create a search bundle. Histograms are used to find the measurement points from the search bundle.
A new estimate for the pose can be calculated from the control points and the corresponding measurement points. The pose estimate is calculated iteratively using the Gauss-Newton algorithm. The 3D model can then be projected to the image plane using the new pose estimate, and the process can be repeated.
The performance of the implemented tracker is evaluated. The tracker can perform the original task that it was supposed to do, but not in all conditions. The tracking may fail if the background is the same color as the phone that is being tracked. Edges in the image that are parallel and close to the edges of the phone may also cause tracking failure. How the phone is grabbed while it is being turned also matters. The tracking may fail if a large part of the phone is not visible to the camera. The phone must not move too much between consecutive processed frames, or the measurement points cannot be found from the next frame, and the tracking fails.
Mathematical concepts that are needed in the implementation of an edge-based tracker are discussed in detail. The first step in the tracking process is to project a 3D model of the object to the image plane. Internal and external matrices are used in perspective projection. The outermost edges of the projected model are found next because the inner edges are not used by the tracker. The outermost edges are divided into control points.
The image from the camera is used to find the corresponding points for the control points. A search line is drawn for each one of the control points. The search lines are perpendicular to the outermost edges of the model. A measurement point should be located on each of the search lines. The measurement points are located on the edges of the phone that is in the image. The search lines are rotated and stacked on top of each other to create a search bundle. Histograms are used to find the measurement points from the search bundle.
A new estimate for the pose can be calculated from the control points and the corresponding measurement points. The pose estimate is calculated iteratively using the Gauss-Newton algorithm. The 3D model can then be projected to the image plane using the new pose estimate, and the process can be repeated.
The performance of the implemented tracker is evaluated. The tracker can perform the original task that it was supposed to do, but not in all conditions. The tracking may fail if the background is the same color as the phone that is being tracked. Edges in the image that are parallel and close to the edges of the phone may also cause tracking failure. How the phone is grabbed while it is being turned also matters. The tracking may fail if a large part of the phone is not visible to the camera. The phone must not move too much between consecutive processed frames, or the measurement points cannot be found from the next frame, and the tracking fails.