No AccessFull-Length Papers

Robust Neurocontrol for Autonomous Dynamic Soaring

Eric J. Kim

https://orcid.org/0000-0002-5914-811X

Royal Military College of Canada, Kingston, Ontario K7K 7B4, Canada

*Master’s Student, Department of Mechanical and Aerospace Engineering.

Search for more papers by this author

and

Ruben E. Perez

Royal Military College of Canada, Kingston, Ontario K7K 7B4, Canada

†Associate Professor, Department of Mechanical and Aerospace Engineering. Senior Member AIAA.

Search for more papers by this author

Published Online:1 Feb 2023https://doi.org/10.2514/1.G006803

Abstract

The flight endurance of small unmanned aerial vehicles can be significantly extended through the exploitation of naturally occurring wind phenomena. However, due to the limited computational hardware on board such aircraft and the uncertain, stochastic nature of real-world environments, there is a need for efficient and robust strategies that exhibit generalized behavior. In addressing these objectives, recent efforts have explored the use of artificial intelligence training algorithms and neural networks for the design of autonomous control schemes that exploit such wind phenomena. This study incorporates the Neuroevolution of Augmenting Topologies algorithm with domain randomization to train robust neurocontrollers that can control an aircraft along sustained traveling dynamic soaring trajectories in the presence of uncertainties and disturbances. This work presents the developed strategy for integrating robustness in neural network control systems, provides a method for quantifying and comparing robustness, and introduces an approach for identifying the network characteristics that contribute to the evolved robust behavior.

References

[1] Denny M., “Dynamic Soaring: Aerodynamics for Albatrosses,” European Journal of Physics, Vol. 30, No. 1, 2008, pp. 75–84. https://doi.org/10.1088/0143-0807/30/1/008 Crossref Google Scholar
[2] Zhao Y. J., “Optimal Patterns of Glider Dynamic Soaring,” Optimal Control Applications and Methods, Vol. 25, No. 2, 2004, pp. 67–89. https://doi.org/10.1002/oca.739 Crossref Google Scholar
[3] Sachs G., “Minimum Shear Wind Strength Required for Dynamic Soaring of Albatrosses,” Ibis, Vol. 147, No. 1, 2005, pp. 1–10. https://doi.org/10.1111/j.1474-919x.2004.00295.x Crossref Google Scholar
[4] Shaw-Cortez W. E. and Frew E., “Efficient Trajectory Development for Small Unmanned Aircraft Dynamic Soaring Applications,” Journal of Guidance, Control, and Dynamics, Vol. 38, No. 3, 2015, pp. 519–523. https://doi.org/10.2514/1.G000543 Link Google Scholar
[5] Akhtar N., Whidborne J. F. and Cooke A. K., “Real-Time Optimal Techniques for Unmanned Air Vehicles Fuel Saving,” Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, Vol. 226, No. 10, 2012, pp. 1315–1328. https://doi.org/10.1177/0954410011418881 Crossref Google Scholar
[6] Gao C. and Liu H. H. T., “Dubins Path-Based Dynamic Soaring Trajectory Planning and Tracking Control in a Gradient Wind Field,” Optimal Control Applications and Methods, Vol. 38, No. 2, 2017, pp. 147–166. https://doi.org/10.1002/oca.2248 Crossref Google Scholar
[7] Li Z. and Langelaan J. W., “Parameterized Trajectory Planning for Dynamic Soaring,” AIAA SciTech 2020 Forum, AIAA Paper 2020-0856, 2020. https://doi.org/10.2514/6.2020-0856 Link Google Scholar
[8] Notter S., Schimpf F. and Fichter W., “Hierarchical Reinforcement Learning Approach Towards Autonomous Cross-Country Soaring,” AIAA SciTech 2021 Forum, AIAA Paper 2021-2010, 2021. https://doi.org/10.2514/6.2021-2010 Link Google Scholar
[9] Reddy G., Celani A., Sejnowski T. J. and Vergassola M., “Learning to Soar in Turbulent Environments,” Proceedings of the National Academy of Sciences, Vol. 113, No. 33, 2016, pp. E4877–E4884. https://doi.org/10.1073/pnas.1606075113 Crossref Google Scholar
[10] Reddy G., Wong-Ng J., Celani A., Sejnowski T. J. and Vergassola M., “Glider Soaring via Reinforcement Learning in the Field,” Nature, Vol. 562, No. 7726, 2018, pp. 236–239. https://doi.org/10.1038/s41586-018-0533-0 Crossref Google Scholar
[11] Kim S.-H., Lee J., Jung S., Lee H. and Kim Y., “Deep Neural Network-Based Feedback Control for Dynamic Soaring of Unpowered Aircraft,” IFAC-PapersOnLine, Vol. 52, No. 12, 2019, pp. 117–121. https://doi.org/10.1016/j.ifacol.2019.11.079 Crossref Google Scholar
[12] Montella C. and Spletzer J. R., “Reinforcement Learning for Autonomous Dynamic Soaring in Shear Winds,” 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, 2014, pp. 3423–3428, https://ieeexplore.ieee.org/document/6943039. https://doi.org/10.1109/IROS.2014.6943039 Google Scholar
[13] Woodbury T. D., Dunn C. and Valasek J., “Autonomous Soaring Using Reinforcement Learning for Trajectory Generation,” 52nd Aerospace Sciences Meeting, AIAA SciTech Forum, AIAA Paper 2014-0990, 2014. https://doi.org/10.2514/6.2014-0990 Link Google Scholar
[14] Chung J., Lawrance N. and Sukkarieh S., “Learning to Soar: Resource-Constrained Exploration in Reinforcement Learning,” International Journal of Robotics Research, Vol. 34, No. 2, Dec. 2014, pp. 158–172. https://doi.org/10.1177/0278364914553683 Google Scholar
[15] Perez R. E., Arnal J. and Jansen P. W., “Neuro-Evolutionary Control for Optimal Dynamic Soaring,” AIAA SciTech 2020 Forum, AIAA Paper 2020-1946, 2020. https://doi.org/10.2514/6.2020-1946 Link Google Scholar
[16] Stanley K. O. and Miikkulainen R., “Evolving Neural Networks Through Augmenting Topologies,” Evolutionary Computation, Vol. 10, No. 2, 2002, pp. 99–127. https://doi.org/10.1162/106365602320169811 Crossref Google Scholar
[17] Kim E. J. and Perez R. E., “Neuroevolutionary Control for Autonomous Soaring,” Aerospace, Vol. 8, No. 9, 2021, Paper 267. https://doi.org/10.3390/aerospace8090267 Google Scholar
[18] Tobin J., Fong R., Ray A., Schneider J., Zaremba W. and Abbeel P., “Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World,” 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Inst. of Electrical and Electronics Engineers, New York, 2017, pp. 23–30. https://doi.org/10.1109/IROS.2017.8202133 Google Scholar
[19] Rayleigh L., “The Soaring of Birds,” Nature, Vol. 27, No. 701, 1883, pp. 534–535. https://doi.org/10.1038/027534a0 Crossref Google Scholar
[20] Owen A. B., Monte Carlo Theory, Methods and Examples, Vol. 16, 2013, pp. 19–22. Google Scholar
[21] Bøhn E., Coates E. M., Moe S. and Johansen T. A., “Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization,” 2019 International Conference on Unmanned Aircraft Systems (ICUAS), 2019, pp. 523–533, https://ieeexplore.ieee.org/document/8798254. https://doi.org/10.1109/ICUAS.2019.8798254 Google Scholar

Journal of Guidance, Control, and Dynamics, volume 46 issue 5 cover

Volume 46, Number 5May 2023

Metrics

Crossmark

Information

Copyright © 2022 by Eric J. Kim and Ruben E. Perez. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. All requests for copying and permission to reprint should be submitted to CCC at www.copyright.com; employ the eISSN 1533-3884 to initiate your request. See also AIAA Rights and Permissions www.aiaa.org/randp.

Topics

Keywords

Acknowledgments

This research was sponsored by Canadian Defence Academy Research Program grant no. 757734, titled “Enhancing UAV Persistent Surveillance Through AI-Driven Optimal Atmospheric Energy Extraction.” The authors also acknowledge a scholarship from Defence Research and Development Canada that enabled this research as part of a Master’s program.

Digital

Received3 March 2022
Accepted7 December 2022
Published online1 February 2023

Robust Neurocontrol for Autonomous Dynamic Soaring

Abstract

References

What's Popular

Metrics

Information