This paper presents a vision-aided uniformly semiglobally exponentially stable nonlinear observer for estimation of attitude, gyro bias, position, velocity, and specific force of a fixed-wing unmanned aerial vehicle, using measurements from an inertial measurement unit, a global navigation satellite system receiver, and computer vision. The computer vision uses optical flow from consecutive images from a camera together with the continuous epipolar constraint to calculate the scaled body-fixed linear velocity, namely, the direction of travel. Epipolar geometry eliminates dependency on the distance to objects in the images and the structure of the terrain being recorded, meaning there is no restriction on the types of terrain for which the observer is applicable. Experimental data from an unmanned aerial vehicle test flight and simulated data are presented, showing that the proposed nonlinear observer has robust performance. Experimental results are compared with an extended Kalman filter and illustrate that the estimates of the states converge to the correct values.

References

[1] Mahony R., Hamel T. and Pflimlin J. M., “Nonlinear Complementary Filters on the Special Orthogonal Group,” IEEE Transactions on Automatic Control, Vol. 53, No. 5, 2008, pp. 1203–1218. doi:https://doi.org/10.1109/TAC.2008.923738 IETAA9 0018-9286 Crossref Google Scholar
[2] Crassidis J. L., Markley F. L. and Cheng Y., “Survey of Nonlinear Attitude Estimation Methods,” Journal of Guidance, Control, and Dynamics, Vol. 30, No. 1, 2007, pp. 12–28. doi:https://doi.org/10.2514/1.22452 JGCODS 0731-5090 Link Google Scholar
[3] Fusini L., Fossen T. I. and Johansen T. A., “A Uniformly Semiglobally Exponentially Stable Nonlinear Observer for GNSS- and Camera-Aided Inertial Navigation,” 22nd IEEE Mediterranean Conference on Control and Automation (MED’14), IEEE Publ., Piscataway, NJ, 2014, pp. 1031–1036. doi:https://doi.org/10.1109/MED.2014.6961510 Google Scholar
[4] Fusini L., Hosen J., Helgesen H., Johansen T. and Fossen T., “Experimental Validation of a Uniformly Semi-Globally Exponentially Stable Non-Linear Observer for GNSS- and Camera-Aided Inertial Navigation for Fixed-Wing UAVs,” International Conference on Unmanned Aircraft Systems (ICUAS), 2015, Inst. of Electrical and Electronics Engineers Inc., New York, 2015, pp. 851–860. doi:https://doi.org/10.1109/ICUAS.2015.7152371 Google Scholar
[5] Hua M. D., “Attitude Estimation for Accelerated Vehicles Using {GPS/INS} Measurements,” Control Engineering Practice, Vol. 18, No. 7, 2010, pp. 723–732. doi:https://doi.org/10.1016/j.conengprac.2010.01.016 COEPEL 0967-0661 Crossref Google Scholar
[6] Grip H. F., Fossen T. I., Johansen T. A. and Saberi A., “A Nonlinear Observer for Integration of {GNSS and IMU} Measurements with Gyro Bias Estimation,” American Control Conference (ACC), Inst. of Electrical and Electronics Engineers Inc., New York, 2012, pp. 4607–4612. doi:https://doi.org/10.1109/ACC.2012.6314929 Google Scholar
[7] Grip H. F., Fossen T. I., Johansen T. A. and Saberi A., “Nonlinear Observer for {GNSS}-Aided Inertial Navigation with Quaternion-Based Attitude Estimation,” American Control Conference (ACC), Inst. of Electrical and Electronics Engineers Inc., New York, 2013, pp. 272–279. doi:https://doi.org/10.1109/ACC.2013.6579849 Google Scholar
[8] Grip H. F., Fossen T. I., Johansen T. A. and Saberi A., “Globally Exponentially Stable Attitude and Gyro Bias Estimation with Application to {GNSS/INS} Integration,” Automatica, Vol. 51, Jan. 2015, pp. 158–166. doi:https://doi.org/10.1016/j.automatica.2014.10.076 ATCAA9 0005-1098 Crossref Google Scholar
[9] Fossen T. I., Handbook of Marine Craft Hydrodynamics and Motion Control, John Wiley & Sons, Chichester, West Sussex, 2011, pp. 15–44. Google Scholar
[10] Euston M., Coote P., Mahony R., Kim J. and Hamel T., “A Complementary Filter for Attitude Estimation of a Fixed-Wing UAV,” 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, IEEE Publ., Piscataway, NJ, 2008, pp. 340–345. doi:https://doi.org/10.1109/IROS.2008.4650766 Google Scholar
[11] Mammarella M., Campa G., Fravolini M. L. and Napolitano M. R., “Comparing Optical Flow Algorithms Using 6-DOF Motion of Real-World Rigid Objects,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 6, 2012, pp. 1752–1762. doi:https://doi.org/10.1109/TSMCC.2012.2218806 Crossref Google Scholar
[12] Zingg S., Scaramuzza D., Weiss S. and Siegwart R., “MAV Navigation through Indoor Corridors Using Optical Flow,” IEEE International Conference on Robotics and Automation, Inst. of Electrical and Electronics Engineers, New York, 2010, pp. 3361–3368. doi:https://doi.org/10.1109/ROBOT.2010.5509777 Google Scholar
[13] Shen S., Michael N. and Kumar V., “Autonomous Multi-Floor Indoor Navigation with a Computationally Constrained MAV,” IEEE International Conference on Robotics and Automation, Inst. of Electrical and Electronics Engineers, New York, pp. 20–25. doi:https://doi.org/10.1109/ICRA.2011.5980357 Google Scholar
[14] Dusha D., Boles W. and Walker R., “Attitude Estimation for a Fixed-Wing Aircraft Using Horizon Detection and Optical Flow,” Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications, Australian Pattern Recognition Soc. (APRS), Glenelg, SA, 2007, pp. 485–492. doi:https://doi.org/10.1109/DICTA.2007.4426836 Google Scholar
[15] Weiss S., Brockers R. and Matthies L., “4DoF Drift Free Navigation Using Inertial Cues and Optical Flow,” IEEE International Conference on Intelligent Robots and Systems, Inst. of Electrical and Electronics Engineers, New York, 2013, pp. 4180–4186. doi:https://doi.org/10.1109/IROS.2013.6696955 Google Scholar
[16] Zufferey J. C. and Floreano D., “Toward 30-gram Autonomous Indoor Aircraft: Vision-Based Obstacle Avoidance and Altitude Control,” IEEE International Conference on Robotics and Automation, Vol. 2005, Inst. of Electrical and Electronics Engineers, New York, 2005, pp. 2594–2599. doi:https://doi.org/10.1109/ROBOT.2005.1570504 Google Scholar
[17] Hrabar S., Sukhatme G. S., Corke P., Usher K. and Roberts J., “Combined Optic-Flow and Stereo-Based Navigation of Urban Canyons for a UAV,” IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2005, pp. 302–309. doi:https://doi.org/10.1109/IROS.2005.1544998 Google Scholar
[18] Merrell P. C., Lee D.-J. and Beard R. W., “Obstacle Avoidance for Unmanned Air Vehicles Using Optical Flow Probability Distributions,” Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 5609, No. 1, 2004, pp. 13–22. doi:https://doi.org/10.1117/12.571554 Google Scholar
[19] Conroy J., Gremillion G., Ranganathan B. and Humbert J. S., “Implementation of Wide-Field Integration of Optic Flow for Autonomous Quadrotor Navigation,” Autonomous Robots, Vol. 27, No. 3, 2009, pp. 189–198. doi:https://doi.org/10.1007/s10514-009-9140-0 Crossref Google Scholar
[20] Ruffier F. and Franceschini N., “Visually Guided Micro-Aerial Vehicle: Automatic Take Off, Terrain Following, Landing and Wind Reaction,” IEEE International Conference on Robotics and Automation, 2004, Vol. 3, Inst. of Electrical and Electronics Engineers, New York, April 2004, pp. 2339–2346. doi:https://doi.org/10.1109/ROBOT.2004.1307411 Google Scholar
[21] Merrell P. C., Lee D.-J. and Beard R. W., “Statistical Analysis of Multiple Optical Flow Values for Estimation of Unmanned Aerial Vehicle Height above Ground,” Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 5608, 2004, pp. 298–305. doi:https://doi.org/10.1117/12.571544 Google Scholar
[22] Brockers R., Susca S., Zhu D. and Matthies L., “Fully Self-Contained Vision-Aided Navigation and Landing of a Micro Air Vehicle Independent from External Sensor Inputs,” Proceedings of SPIE—The International Society for Optical Engineering, Vol. 8387, 2012, Paper 83870Q. doi:https://doi.org/10.1117/12.919278 Google Scholar
[23] Herisse B., Russotto F. X., Hamel T. and Mahony R., “Hovering Flight and Vertical Landing Control of a VTOL Unmanned Aerial Vehicle Using Optical Flow,” 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, IEEE Publ., Piscataway, NJ, 2008, pp. 801–806. doi:https://doi.org/10.1109/IROS.2008.4650731 Google Scholar
[24] Kehoe J. J., Watkins A. S., Causey R. S. and Lind R., “State Estimation Using Optical Flow from Parallax-Weighted Feature Tracking,” Collection of Technical Papers/AIAA Guidance, Navigation, and Control Conference, Vol. 8, 2006, pp. 5030–5045. doi:https://doi.org/10.2514/6.2006-6721 Google Scholar
[25] Moore R. J. D., Thurrowgood S. and Srinivasan M. V., “Vision-Only Estimation of Wind Field Strength and Direction from an Aerial Platform,” IEEE International Conference on Intelligent Robots and Systems, Inst. of Electrical and Electronics Engineers, New York, 2012, pp. 4544–4549. doi:https://doi.org/10.1109/IROS.2012.6385682 Google Scholar
[26] Weiss S., Achtelik M. W., Lynen S., Chli M. and Siegwart R., “Real-Time Onboard Visual-Inertial State Estimation and Self-Calibration of MAVs in Unknown Environments,” IEEE International Conference on Robotics and Automation, Inst. of Electrical and Electronics Engineers, New York, 2012, pp. 957–964. doi:https://doi.org/10.1109/ICRA.2012.6225147 Google Scholar
[27] Mercado D. A., Flores G., Castillo P., Escareno J. and Lozano R., “GPS/INS/Optic Flow Data Fusion for Position and Velocity Estimation,” International Conference on Unmanned Aircraft Systems, ICUAS, IEEE Publ., Piscataway, NJ, 2013, pp. 486–491. doi:https://doi.org/10.1109/ICUAS.2013.6564724 Google Scholar
[28] Bibuli M., Caccia M. and Lapierre L., “Path-Following Algorithms and Experiments for an Autonomous Surface Vehicle,” Control Applications in Marine Systems, Vol. 7, No. Part 1, 2007, pp. 81–86. doi:https://doi.org/10.3182/20070919-3-HR-3904.00015 Google Scholar
[29] Ahrens S., Levine D., Andrews G. and How J. P., “Vision-Based Guidance and Control of a Hovering Vehicle in Unknown, GPS-Denied Environments,” IEEE International Conference on Robotics and Automation, Inst. of Electrical and Electronics Engineers, New York, 2009, pp. 2643–2648. doi:https://doi.org/10.1109/ROBOT.2009.5152680 Google Scholar
[30] Batista P., Silvestre C. and Oliveira P., “{GES} Attitude Observers—{Part I}: Multiple General Vector Observations,” 18th IFAC World Congress, Vol. 18, International Federation of Automatic Control, 2011, pp. 2985–2990. Google Scholar
[31] Batista P., Silvestre C. and Oliveira P., “{GES} Attitude Observers—{Part II}: Single Vector Observations,” 18th IFAC World Congress, Vol. 18, International Federation of Automatic Control, 2011, pp. 2991–2996. Google Scholar
[32] Grip H. F., Fossen T. I., Johansen T. A. and Saberi A., “Attitude Estimation Using Biased Gyro and Vector Measurements with Time-Varying Reference Vectors,” IEEE Transactions on Automatic Control, Vol. 57, No. 5, 2012, pp. 1332–1338. doi:https://doi.org/10.1109/TAC.2011.2173415 IETAA9 0018-9286 Crossref Google Scholar
[33] Guerrero-Castellanos J., Madrigal-Sastre H., Durand S., Torres L. and Muñoz Hernández G., “A Robust Nonlinear Observer for Real-Time Attitude Estimation Using Low-Cost {MEMS} Inertial Sensors,” Sensors, Vol. 13, No. 11, 2013, pp. 15138–15158. doi:https://doi.org/10.3390/s131115138 SNSRES 0746-9462 Crossref Google Scholar
[34] Hua M. D., Ducard G., Hamel T., Mahony R. and Rudin K., “Implementation of a Nonlinear Attitude Estimator for Aerial Robotic Vehicles,” IEEE Transactions on Control System Technology, Vol. 22, No. 1, 2014, pp. 201–213. doi:https://doi.org/10.1109/TCST.2013.2251635 Crossref Google Scholar
[35] Mahony R., Hamel T., Trumpf J. and Lageman C., “Nonlinear Attitude Observers on {SO(3)} for Complementary and Compatible Measurements: A Theoretical Study,” 48th IEEE Conference on Decision and Control, Inst. of Electrical and Electronics Engineers, New York, 2009, pp. 6407–6412. doi:https://doi.org/10.1109/CDC.2009.5399821 Google Scholar
[36] Mahony R., Euston M., Kim J., Coote1 P. and Hamel T., “A Non-Linear Observer for Attitude Estimation of a Fixed-Wing Unmanned Aerial Vehicle without {GPS} Measurements,” Transactions of the Institute of Measurement and Control, Vol. 33, No. 6, 2011, pp. 699–717. doi:https://doi.org/10.1177/0142331209343660 TICODG 0142-3312 Crossref Google Scholar
[37] Salcudean S., “A Globally Convergent Angular Velocity Observer for Rigid Body Motion,” IEEE Transactions on Automatic Control, Vol. 36, No. 12, 1991, pp. 1493–1497. IETAA9 0018-9286 Crossref Google Scholar
[38] Thienel J. and Sanner R. M., “A Coupled Nonlinear Spacecraft Attitude Controller and Observer with an Unknown Constant Gyro Bias and Gyro Noise,” IEEE Transactions on Automatic Control, Vol. 48, No. 11, 2003, pp. 2011–2015. doi:https://doi.org/10.1109/TAC.2003.819289 IETAA9 0018-9286 Crossref Google Scholar
[39] Lucas B. D. and Kanade T., “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI 81), Morgan Kaufmann Publishers Inc., San Francisco, CA, 1981, pp. 674–679. Google Scholar
[40] Horn B. K. and Schunck B. G., “Determining Optical Flow,” Artificial Intelligence, Vol. 17, Nos. 1–3, 1981, pp. 185–203. doi:https://doi.org/10.1016/0004-3702(81)90024-2 AINTBB 0004-3702 Crossref Google Scholar
[41] Lowe D., “Object Recognition from Local Scale-Invariant Features,” Seventh IEEE International Conference on Computer Vision, Vol. 2, Inst. of Electrical and Electronics Engineers, New York, 1999, pp. 1150–1157. doi:https://doi.org/10.1109/ICCV.1999.790410 Google Scholar
[42] Bay H., Ess A., Tuytelaars T. and Van Gool L., “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, Vol. 110, No. 3, 2008, pp. 346–359. doi:https://doi.org/10.1016/j.cviu.2007.09.014 Google Scholar
[43] Diel D. D., DeBitetto P. and Teller S., “Epipolar Constraints for Vision-Aided Inertial Navigation,” IEEE Workshop on Application of Computer Vision, Vol. 2, Inst. of Electrical and Electronics Engineers, New York, 2005, pp. 221–228. doi:https://doi.org/10.1109/ACVMOT.2005.48 Google Scholar
[44] Bazin J. C., Demonceaux C., Vasseur P. and Kweon I. S., “Motion Estimation by Decoupling Rotation and Translation in Catadioptric Vision,” Computer Vision and Image Understanding, Vol. 114, No. 2, 2010, pp. 254–273. doi:https://doi.org/10.1016/j.cviu.2009.04.006 CVIUF4 1077-3142 Crossref Google Scholar
[45] Meingast M., Geyer C. and Sastry S., “Vision Based Terrain Recovery for Landing Unmanned Aerial Vehicles,” 43rd IEEE Conference on Decision and, Vol. 2, Inst. of Electrical and Electronics Engineers, New York, 2004, pp. 1670–1675. doi:https://doi.org/10.1109/CDC.2004.1430284 Google Scholar
[46] Sanfourche M., Delaune J. and Besnerais G., “Perception for UAV: Vision-Based Navigation and Environment Modeling,” Aerospace Lab Journal, Vol. 4, 2012, pp. 1–19. Google Scholar
[47] Davison A. J., Reid I. D., Molton N. D. and Stasse O., “MonoSLAM: Real-Time Single Camera SLAM,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 6, 2007, pp. 1052–1067. doi:https://doi.org/10.1109/TPAMI.2007.1049 ITPIDJ 0162-8828 Crossref Google Scholar
[48] Ma Y., Soatto S., Košecká J. and Sastry S., An Invitation to 3D Vision, Springer–Verlag, New York, 2004, pp. 109–170. doi:https://doi.org/10.1006/ndsh.1995.1003 Crossref Google Scholar
[49] Grabe V., Bulthoff H. H. and Robuffo Giordano P., “Robust Optical-Flow Based Self-Motion Estimation for a Quadrotor UAV,” IEEE International Conference on Intelligent Robots and Systems, Inst. of Electrical and Electronics Engineers, New York, 2012, pp. 2153–2159. doi:https://doi.org/10.1109/IROS.2012.6386234 Google Scholar
[50] Grabe V., Bulthoff H. H. and Giordano P. R., “On-Board Velocity Estimation and Closed-Loop Control of a Quadrotor UAV Based on Optical Flow,” 2012 IEEE International Conference on Robotics and Automation, Inst. of Electrical and Electronics Engineers, New York, 2012, pp. 491–497. doi:https://doi.org/10.1109/ICRA.2012.6225328 Google Scholar
[51] Hartley R. and Zisserman A., Multiple View Geometry in Computer Vision, Press Syndicate of Univ. of Cambridge, Cambridge, U.K., 2003, pp. 239–261. Google Scholar
[52] Shi J. and Tomasi C., “Good Features to Track,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Inst. of Electrical and Electronics Engineers, New York, 1994, pp. 593–600. doi:https://doi.org/10.1109/CVPR.1994.323794 Google Scholar
[53] Muja M. and Lowe D. G., “Flann, Fast Library for Approximate Nearest Neighbors,” International Conference on Computer Vision Theory and Applications (VISAPP’09), INSTICC Press, 2009. Google Scholar
[54] Fuh C. S. and Maragos P., “Region-Based Optical Flow Estimation,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Inst. of Electrical and Electronics Engineers, New York, 1989, pp. 130–135. doi:https://doi.org/10.1109/CVPR.1989.37840 Google Scholar
[55] Sarvaiya J. N., Patnaik S. and Bombaywala S., “Image Registration by Template Matching Using Normalized Cross-Correlation,” International Conference on Advances in Computing, Control and Telecommunication Technologies, IEEE Publ., Piscataway, NJ, 2009, pp. 819–822. doi:https://doi.org/10.1109/ACT.2009.207 Google Scholar
[56] Hutchinson S., Hager G. D. and Corke P. I., “A Tutorial on Visual Servo Control,” IEEE Transactions on Robotics and Automation, Vol. 12, No. 5, 1996, pp. 651–670. IRAUEZ 1042-296X Crossref Google Scholar
[57] Kreyszig E., Advanced Engineering Mathematics, Vol. 53, Wiley, 2006, pp. 364–419. doi:https://doi.org/10.2307/3612523. Google Scholar
[58] Krstic M., Kanellakopoulos I. and Kokotovic P. V., Nonlinear and Adaptive Control Design, Wiley, New York, 1995, pp. 511–515. Google Scholar
[59] Shuster M. D. and Oh S. D., “Three-Axis Attitude Determination from Vector Observations,” Journal of Guidance, Control, and Dynamics, Vol. 4, No. 1, 1981, pp. 70–77. doi:https://doi.org/10.2514/3.19717 Link Google Scholar
[60] Bradski G., “The Open CV Library,” Dr. Dobbs Journal of Software Tools, Vol. 25, No. 11, 2000, pp. 120–125. doi:https://doi.org/10.1111/0023-8333.50.s1.10 Google Scholar

Journal of Guidance, Control, and Dynamics, volume 39 issue 8 cover

Volume 39, Number 8August 2016

Metrics

Crossmark

Information

Copyright © 2016 by Jesper Hosen. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. Copies of this paper may be made for personal and internal use, on condition that the copier pay the per-copy fee to the Copyright Clearance Center (CCC). All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the ISSN 0731-5090 (print) or 1533-3884 (online) to initiate your request.

Topics

Keywords

Acknowledgments

The authors are grateful for the assistance provided by the unmanned aerial vehicle engineers at the Norwegian University of Science and Technology and Maritime Robotics, in particular, Lars Semb and Carl Erik Stephansen. Significant contributions to the construction of the unmanned aerial vehicle payload was made by the rest of the navigation team at Norwegian University of Science and Technology, in particular, Sigurd M. Albrektsen, Jakob M. Hansen, and Kasper T. Borup. This work was partly supported by the Norwegian Research Council (grant numbers 225259 and 223254) through the Center of Autonomous Marine Operations and Systems at Norwegian University of Science and Technology.

Digital

Received25 November 2015
Accepted18 February 2016
Published online3 June 2016

Vision-Aided Nonlinear Observer for Fixed-Wing Unmanned Aerial Vehicle Navigation

Abstract

References

What's Popular

Metrics

Information