Skip to main content
Skip to article control options
No AccessFull-Length Papers

Particle Filter Approach to Vision-Based Navigation with Aerial Image Segmentation

Published Online:https://doi.org/10.2514/1.I010957

This study proposes a novel approach for a vision-based navigation problem using semantically segmented aerial images generated by a convolutional neural network. Vision-based navigation provides a position solution by matching an aerial image to a georeferenced database, and it has been increasingly studied for global navigation satellite system–denied environments. Aerial images include a vast amount of information that infers the position where they are located. However, it also includes features that disturb the estimation accuracy. The progress of convolutional neural network may provide a promising solution for extracting only helpful features for this purpose. Therefore, segmented images are modeled as a Gaussian mixture model, and the L2 distance for a quantitative discrepancy between two images is established. This allows us to compare the two images quickly with improved accuracy. In addition, a framework of a particle filter is applied to estimate the position using an inertial navigation system. It employs the L2 distance as a measurement, and the particles tend to converge to the true position. Flight test experiments were conducted to verify that the proposed approach achieved distance error of less than 10 m.

References

  • [1] Xu Y., Pan L., Du C., Li J., Jing N. and Wu J., “Vision-Based UAVs Aerial Image Localization: A Survey,” Proceedings of the 2nd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, Assoc. for Computing Machinery, New York, Nov. 2018, pp. 9–18. https://doi.org/10.1145/3281548.3281556 Google Scholar

  • [2] Conte G. and Doherty P., “Vision-Based Unmanned Aerial Vehicle Navigation Using Geo-Referenced Information,” EURASIP Journal on Advances in Signal Processing, Vol. 2009, Jan. 2009, pp. 1–18. https://doi.org/10.1155/2009/387308 Google Scholar

  • [3] Koch T., Zhuo X., Reinartz P. and Fraundorfer F., “A New Paradigm for Matching UAV and Aerial Images,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 3, July 2016, pp. 83–90. https://doi.org/10.5194/isprs-annals-III-3-83-2016 CrossrefGoogle Scholar

  • [4] Long J., Shelhamer E. and Darrell T., “Fully Convolutional Networks for Semantic Segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, New York, 2015, pp. 3431–3440. Google Scholar

  • [5] He K., Gkioxari G., Dollár P. and Girshick R., “Mask R-CNN,” Proceedings of the IEEE International Conference on Computer Vision, IEEE, New York, 2017, pp. 2961–2969. Google Scholar

  • [6] Tian Y., Chen C. and Shah M., “Cross-View Image Matching for Geo-Localization in Urban Environments,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, New York, 2017, pp. 3608–3616. Google Scholar

  • [7] Hong K., Kim S. and Bang H., “Vision-Based Navigation Using Gaussian Mixture Model of Terrain Features,” AIAA Scitech 2020 Forum, AIAA Paper 2020-1344, Jan. 2020. https://doi.org/10.2514/6.2020-1344 Google Scholar

  • [8] Nassar A., Amer K., ElHakim R. and ElHelw M., “A Deep CNN-Based Framework for Enhanced Aerial Imagery Registration with Applications to UAV Geolocalization,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, New York, 2018, pp. 1513–1523. Google Scholar

  • [9] Goforth H. and Lucey S., “GPS-Denied UAV Localization Using Pre-Existing Satellite Imagery,” 2019 International Conference on Robotics and Automation (ICRA), IEEE, New York, May 2019, pp. 2974–2980. Google Scholar

  • [10] Choi J. and Myung H., “BRM Localization: UAV Localization in GNSS-Denied Environments Based on Matching of Numerical Map and UAV Images,” arXiv preprint arXiv:2008.01347. Google Scholar

  • [11] Arulampalam M. S., Maskell S., Gordon N. and Clapp T., “A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking,” IEEE Transactions on Signal Processing, Vol. 50, No. 2, Aug. 2002, pp. 174–188. https://doi.org/10.1109/78.978374 CrossrefGoogle Scholar

  • [12] Bhat D. N. and Nayar S. K., “Ordinal Measures for Image Correspondence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 4, April 1998, pp. 415–423. https://doi.org/10.1109/34.677275 CrossrefGoogle Scholar

  • [13] Lowe D. G., “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, Vol. 60, No. 2, Nov. 2004, pp. 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94 CrossrefGoogle Scholar

  • [14] Bay H., Ess A., Tuytelaars T. and Van Gool L., “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, Vol. 110, No. 3, June 2008, pp. 346–359. https://doi.org/10.1016/j.cviu.2007.09.014 CrossrefGoogle Scholar

  • [15] Rublee E., Rabaud V., Konolige K. and Bradski G., “ORB: An Efficient Alternative to SIFT or SURF,” 2011 International Conference on Computer Vision, IEEE, New York, Nov. 2011, pp. 2564–2571. https://doi.org/10.1109/ICCV.2011.6126544 Google Scholar

  • [16] Noble F. K., “Comparison of OpenCV’s Feature Detectors and Feature Matchers,” 2016 23rd International Conference on Mechatronics and Machine Vision in Practice (M2VIP), IEEE, New York, Nov. 2016, pp. 1–6. https://doi.org/10.1109/M2VIP.2016.7827292 Google Scholar

  • [17] Ahrendt P., “The Multivariate Gaussian Probability Distribution,” Tech. Rept., Technical Univ. of Denmark, 2005. Google Scholar

  • [18] Rogers R. M., Applied Mathematics in Integrated Navigation Systems, AIAA, Reston, VA, 2007, Chap. 14. https://doi.org/10.2514/4.861598 LinkGoogle Scholar

  • [19] Ester M., Kriegel H. P., Sander J. and Xu X., “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise,” KDD 1996 Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD-96, Vol. 96, AAAI Press, Menlo Park, CA, 1996, pp. 226–231. Google Scholar

  • [20] Douc R. and Cappé O., “Comparison of Resampling Schemes for Particle Filtering,” ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, IEEE, New York, Sept. 2005, pp. 64–69. https://doi.org/10.1109/ISPA.2005.195385 Google Scholar