Skip to main content
Search
Search
Quick Search anywhere
Quick Search fdjslkfh
Advanced search
Cart
Join AIAA Institution Login
Skip main navigationClose Drawer MenuOpen Drawer Menu
Journal of Guidance, Control, and Dynamics logo
No AccessFull-Length Papers

Optimal Nonlinear Control Using Hamilton–Jacobi–Bellman Viscosity Solutions on Unstructured Grids

Published Online:7 Nov 2019https://doi.org/10.2514/1.G004362

Abstract

The optimal control of nonlinear systems is traditionally obtained by the application of the Pontryagin minimum principle. Despite the success of this methodology in finding the optimal control for complex systems, the resulting open-loop trajectory is guaranteed to be only locally optimal. Furthermore, the computation of open-loop solutions is computationally intensive, which rules out its application in feedback controllers, reducing its robustness against disturbances. In principle, these issues can be addressed by solving the Hamilton–Jacobi–Bellman (HJB) partial differential equation (PDE). However, the space complexity of the problem is exponential with respect to the number of dimensions of the system. Moreover, the value function of the HJB equation may be nondifferentiable, which renders traditional PDE solution methods impractical. Therefore, extant methods are suitable only for special problem classes such as those involving affine systems or where the value function is differentiable. To deal with these issues, this work introduces a methodology for the solution of the HJB equation for general nonlinear systems that combines PDE viscosity solutions, quasi–Monte Carlo grids, and kriging regression to implement globally optimal nonlinear feedback controllers for practical applications. The effectiveness of the method is illustrated with smooth and nondifferentiable problems with finite and infinite horizons.

References

  • [1] Bryson A. E. and Ho Y. C., Applied Optimal Control, Hemisphere, New York, 1975, Chaps. 2, 4, 5, 6. Google Scholar

  • [2] Conway B. A. and Paris S. W., “Spacecraft Trajectory Optimization Using Direct Transcription and Nonlinear Programming,” Spacecraft Trajectory Optimization, edited by Conway B. A., Cambridge Univ. Press, New York, 2010. CrossrefGoogle Scholar

  • [3] Chilan C. M. and Conway B. A., “A Reachable Set Analysis Method for Generating Near-Optimal Trajectories of Constrained Multiphase Systems,” Journal of Optimization Theory and Applications, Vol. 167, No. 1, 2015, pp. 161–194. https://doi.org/10.1007/s10957-014-0651-2 CrossrefGoogle Scholar

  • [4] Pontryagin L. S., Boltyanskii V. G., Gamkrelidze R. V. and Mishchenko E. F., The Mathematical Theory of Optimal Processes, Interscience, New York, 1962, Chap. 1. Google Scholar

  • [5] Chilan C. M., Conway B. A., Bialy B. J. and Stockbridge S., “Optimal Nonlinear Feedback with Feed Forward Control of High Speed Aerospace Vehicles Using a Spatial Statistical Approach,” AAS/AIAA Astrodynamics Specialist Conference, AAS Paper 18-215, Springfield, VA, 2018. Google Scholar

  • [6] Ghosh P. and Conway B. A., “Near-Optimal Feedback Strategies Synthesized Using a Spatial Statistical Approach,” Journal of Guidance, Control, and Dynamics, Vol. 36, No. 4, 2013, pp. 905–919. https://doi.org/10.2514/1.59568 LinkGoogle Scholar

  • [7] Jardin M. R. and Bryson A. E., “Methods for Computing Minimum-Time Paths in Strong Winds,” Journal of Guidance, Control, and Dynamics, Vol. 35, No. 1, 2012, pp. 165–171 https://doi.org/10.2514/1.53614 LinkGoogle Scholar

  • [8] Kalman R. E., “The Theory of Optimal Control and the Calculus of Variations,” Mathematical Optimization Techniques, edited by Bellman R., Univ. of California Press, Berkeley, CA, 1963. Google Scholar

  • [9] Bellman R. E. and Dreyfus S. E., Applied Dynamic Programming, Princeton Univ. Press, Princeton, NJ, 1962, Chaps. 1, 12. CrossrefGoogle Scholar

  • [10] Falcone M., “Numerical Solution of Dynamic Programming Equations,” Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Equations, edited by Bardi M. and Capuzzo-Dolcetta I., Birkhäuser, Boston, 1997, Appendix A. Google Scholar

  • [11] Cristiani E. and Martinon P., “Initialization of the Shooting Method via the Hamilton-Jacobi-Bellman Approach,” Journal of Optimization Theory and Applications, Vol. 146, No. 2, 2010, pp. 321–346. https://doi.org/10.1007/s10957-010-9649-6 CrossrefGoogle Scholar

  • [12] Kang W. and Wilcox L. C., “Mitigating the Curse of Dimensionality: Sparse Grid Characteristics Method for Optimal Feedback Control and HJB Equations,” Computational Optimization and Applications, Vol. 68, No. 2, 2017, pp. 289–315. https://doi.org/10.1007/s10589-017-9910-0 CrossrefGoogle Scholar

  • [13] Bardi M. and Capuzzo Dolcetta I., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhäuser, Boston, 1997, Chaps. 1, 2, 3, 4, 6. CrossrefGoogle Scholar

  • [14] Adurthi N., Singla P. and Majji M., “Sparse Approximation-Based Collocation Scheme for Nonlinear Optimal Feedback Control Design,” Journal of Guidance, Control, and Dynamics, Vol. 40, No. 2, 2017, pp. 248–264. https://doi.org/10.2514/1.G001755 LinkGoogle Scholar

  • [15] Metropolis N. and Ulam S., “The Monte Carlo Method,” Journal of the American Statistical Association, Vol. 44, No. 247, 1949, pp. 335–341. https://doi.org/10.1080/01621459.1949.10483310 CrossrefGoogle Scholar

  • [16] Caflisch R. E., “Monte Carlo and Quasi-Monte Carlo Methods,” Acta Numerica, Vol. 7, Jan. 1998, pp. 1–49. https://doi.org/10.1017/S0962492900002804 CrossrefGoogle Scholar

  • [17] Morokoff W. J. and Caflisch R. E., “Quasi-Monte Carlo Integration,” Journal of Computational Physics, Vol. 122, No. 2, 1995, pp. 218–230. https://doi.org/10.1006/jcph.1995.1209 Google Scholar

  • [18] Krige D. G., “A Statistical Approach to Some Basic Mine Valuation Problems on the Witwatersrand,” Journal of the Chemical, Metallurgical, and Mining Society of South Africa, Vol. 52, No. 6, 1951, pp. 119–139. Google Scholar

  • [19] Lophaven S. N., Nielsen H. B. and Søndergaard J., DACE: A MATLAB Kriging Toolbox, Technical Univ. of Denmark, IMM-TR-2002-12, Lyngby, Denmark, 2002. Google Scholar

  • [20] Crandall M. G. and Lions P.-L., “Viscosity Solutions of Hamilton-Jacobi Equations,” Transactions of the American Mathematical Society, Vol. 277, No. 1, 1983, pp. 1–42. https://doi.org/10.1090/S0002-9947-1983-0690039-8 CrossrefGoogle Scholar

  • [21] Dammertz S. and Keller A., “Image Synthesis by Rank-1 Lattices,” Monte Carlo and Quasi-Monte Carlo Methods 2006, edited by Keller A., Heinrich S. and Niederreiter H., Springer–Verlag, Berlin, 2008. Google Scholar

  • [22] Kocis L. and Whiten W. J., “Computational Investigations of Low-Discrepancy Sequences,” ACM Transactions on Mathematical Software, Vol. 23, No. 2, 1997, pp. 266–294. https://doi.org/10.1145/264029.264064 CrossrefGoogle Scholar

  • [23] Statistics and Machine Learning Toolbox, R2018b, The MathWorks, Natick, MA, 2019, https://www.mathworks.com/products/statistics.html [retrieved 24 July 2019]. Google Scholar

  • [24] Matheron G., “Principles of Geostatistics,” Economic Geology, Vol. 58, No. 8, 1963, pp. 1246–1266. https://doi.org/10.2113/gsecongeo.58.8.1246 CrossrefGoogle Scholar

  • [25] Sacks J., Welch W. J., Mitchell T. J. and Wynn H. P., “Design and Analysis of Computer Experiments,” Statistical Science, Vol. 4, No. 4, 1989, pp. 409–423. https://doi.org/10.1214/ss/1177012413 CrossrefGoogle Scholar

  • [26] Grama A., Gupta A., Karypis G. and Kumar V., Introduction to Parallel Computing, 2nd ed., Addison-Wesley, Harlow, England, U.K., 2003, Chap. 5. Google Scholar

  • [27] Falcone M., “Numerical Methods for Differential Games Based on Partial Differential Equations,” International Game Theory Review, Vol. 8, No. 2, 2006, pp. 231–272. https://doi.org/10.1142/S0219198906000886 Google Scholar

  • [28] Patterson M. A. and Rao A. V., “GPOPS-II: A MATLAB Software for Solving Multiple-Phase Optimal Control Problems Using hp-Adaptive Gaussian Quadrature Collocation Methods and Sparse Nonlinear Programming,” ACM Transactions on Mathematical Software, Vol. 41, No. 1, 2014, pp. 1–37. https://doi.org/10.1145/2558904 CrossrefGoogle Scholar

Previous article
Next article