Skip to main content
Skip to article control options
No AccessRegular Articles

Impact of Number Representation for High-Order Implicit Large-Eddy Simulations

Published Online:

High-order numerical methods for unstructured grids combine the superior accuracy of high-order spectral or finite difference methods with the geometric flexibility of low-order finite volume or finite element schemes. Over the past few years, they have shown promise in enabling implicit large-eddy simulations within the vicinity of complex geometrical configurations. However, the cost of such simulations remains prohibitive. In this paper, the impact of number representation on the efficiency and efficacy of these methods is considered. Theoretical performance models are derived and assessed. Specifically, four test cases are considered: 1) the viscous Taylor–Green vortex, 2) flow over a circular cylinder, 3) flow through a T106c low-pressure turbine cascade, and 4) flow around a NACA 0021 in deep stall. It is found that the use of single (in lieu of double) precision arithmetic does not have a significant impact on accuracy. The performance of the resulting simulations was found to improve between 1.4 and 2.9 times.


  • [1] Reed W. H. and Hill T. R., “Triangular Mesh Methods for the Neutron Transport Equation,” Los Alamos Scientific Lab. TR LA-UR-73-479, Los Alamos, NM, 1973. Google Scholar

  • [2] Kopriva D. A. and Kolias J. H., “A Conservative Staggered-Grid Chebyshev Multidomain Method for Compressible Flows,” Journal of Computational Physics, Vol. 125, No. 1, 1996, pp. 244–261. doi: CrossrefGoogle Scholar

  • [3] Sun Y., Wang Z. J. and Liu Y., “High-Order Multidomain Spectral Difference Method for the Navier–Stokes Equations on Unstructured Hexahedral Grids,” Communications in Computational Physics, Vol. 2, No. 2, 2007, pp. 310–333. Google Scholar

  • [4] Huynh H. T., “A Flux Reconstruction Approach to High-Order Schemes Including Discontinuous Galerkin Methods,” AIAA Paper 2007-4079, 2007. doi: LinkGoogle Scholar

  • [5] Hesthaven J. S. and Warburton T., Nodal Discontinuous Galerkin Methods: Algorithms, Analysis, and Applications, Vol. 54, Springer, New York, 2008. doi: CrossrefGoogle Scholar

  • [6] Klöckner A., Warburton T., Bridge J. and Hesthaven J. S., “Nodal Discontinuous Galerkin Methods on Graphics Processors,” Journal of Computational Physics, Vol. 228, No. 21, 2009, pp. 7863–7882. doi: CrossrefGoogle Scholar

  • [7] Castonguay P., Williams D. M., Vincent P. E., Lopez M. and Jameson A., “On the Development of a High-Order, Multi-GPU Enabled, Compressible Viscous Flow Solver for Mixed Unstructured Grids,” AIAA Paper 2011-3229, 2011. doi: Google Scholar

  • [8] Witherden F. D., Farrington A. M. and Vincent P. E., “PyFR: An Open Source Framework for Solving Advection–Diffusion Type Problems on Streaming Architectures Using the Flux Reconstruction Approach,” Computer Physics Communications, Vol. 185, No. 11, 2014, pp. 3028–3040. doi: CrossrefGoogle Scholar

  • [9] Witherden F. D., Vermeire B. C. and Vincent P. E., “Heterogeneous Computing on Mixed Unstructured Grids with PyFR,” Computers and Fluids, Vol. 120, Oct. 2015, pp. 173–186. doi: CrossrefGoogle Scholar

  • [10] IEEE Standard for Floating-Point Arithmetic,” IEEE STD 754-2008, IEEE Publ., Piscataway, NJ, Aug. 2008, pp. 1–70. doi: Google Scholar

  • [11] Bailey D. H., “High-Precision Floating-Point Arithmetic in Scientific Computation,” Computing in Science and Engineering, Vol. 7, No. 3, 2005, pp. 54–61. doi: CrossrefGoogle Scholar

  • [12] Abraham M. J., Murtola T., Schulz R., Páll S., Smith J. C., Hess B. and Lindahl E., “GROMACS: High Performance Molecular Simulations Through Multi-Level Parallelism from Laptops to Supercomputers,” SoftwareX, Vols. 1–2, Sept. 2015, pp. 19–25. doi: CrossrefGoogle Scholar

  • [13] Board J. A., Causey J. W., Leathrum J. F., Windemuth A. and Schulten K., “Accelerated Molecular Dynamics Simulation with the Parallel Fast Multipole Algorithm,” Chemical Physics Letters, Vol. 198, Nos. 1–2, 1992, pp. 89–94. doi: Google Scholar

  • [14] Hess B., Kutzner C., Van Der Spoel D. and Lindahl E., “GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation,” Journal of Chemical Theory and Computation, Vol. 4, No. 3, 2008, pp. 435–447. doi: CrossrefGoogle Scholar

  • [15] Heinecke A., Breuer A., Rettenberger S., Bader M., Gabriel A.-A., Pelties C., Bode A., Barth W., Liao X.-K., Vaidyanathan K. and et al., “Petascale High Order Dynamic Rupture Earthquake Simulations on Heterogeneous Supercomputers,” Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE Publ., Piscataway, NJ, 2014, pp. 3–0. doi: Google Scholar

  • [16] Gasperini P. and Vannucci G., “FPSPACK: A Package of FORTRAN Subroutines to Manage Earthquake Focal Mechanism Data,” Computers and Geosciences, Vol. 29, No. 7, 2003, pp. 893–901. doi: Google Scholar

  • [17] Komatitsch D., Michéa D. and Erlebacher G., “Porting a High-Order Finite-Element Earthquake Modeling Application to NVIDIA Graphics Cards Using CUDA,” Journal of Parallel and Distributed Computing, Vol. 69, No. 5, 2009, pp. 451–460. doi: Google Scholar

  • [18] Homann H., Dreher J. and Grauer R., “Impact of the Floating-Point Precision and Interpolation Scheme on the Results of DNS of Turbulence by Pseudo-Spectral Codes,” Computer Physics Communications, Vol. 177, No. 7, 2007, pp. 560–565. doi: CrossrefGoogle Scholar

  • [19] Amdahl G. M., “Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities,” Proceedings of the April 18-20, 1967, Spring Joint Computer Conference, ACM Press, New York, 1967, pp. 483–485. doi: Google Scholar

  • [20] Lai J. and Seznec A., “Performance Upper Bound Analysis and Optimization of SGEMM on Fermi and Kepler GPUs,” 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), IEEE Publ., Piscataway, NJ, 2013, pp. 1–10. doi: Google Scholar

  • [21] Cockburn B. and Shu C.-W., “The Local Discontinuous Galerkin Method for Time-Dependent Convection-Diffusion Systems,” SIAM Journal on Numerical Analysis, Vol. 35, No. 6, 1998, pp. 2440–2463. doi: CrossrefGoogle Scholar

  • [22] Toro E. F., Riemann Solvers and Numerical Methods for Fluid Dynamics: A Practical Introduction, Springer, New York, 2009. doi: CrossrefGoogle Scholar

  • [23] Park J., Witherden F. and Vincent P., “High-Order Implicit Large-Eddy Simulations of Flow over a NACA0021 Aerofoil,” AIAA Journal, Vol. 55, No. 7, June 2017, pp. 2186–2197. doi: LinkGoogle Scholar

  • [24] Witherden F. D., “On the Development and Implementation of High-Order Flux Reconstruction Schemes for Computational Fluid Dynamics,” Ph.D. Thesis, Imperial College London, 2015. Google Scholar

  • [25] Witherden F. and Vincent P., “On the Identification of Symmetric Quadrature Rules for Finite Element Methods,” Computers and Mathematics with Applications, Vol. 69, No. 10, 2015, pp. 1232–1241. doi: CrossrefGoogle Scholar

  • [26] Shunn L. and Ham F., “Symmetric Quadrature Rules for Tetrahedra Based on a Cubic Close-Packed Lattice Arrangement,” Journal of Computational and Applied Mathematics, Vol. 236, No. 17, 2012, pp. 4348–4364. doi: CrossrefGoogle Scholar

  • [27] Williams D. M., Shunn L. and Jameson A., “Symmetric Quadrature Rules for Simplexes Based on Sphere Close Packed Lattice Arrangements,” Journal of Computational and Applied Mathematics, Vol. 266, Aug. 2014, pp. 18–38. doi: CrossrefGoogle Scholar

  • [28] Vincent P. E., Witherden F. D., Farrington A. M., Ntemos G., Vermeire B. C., Park J. S. and Iyer A. S., “PyFR: Next-Generation High-Order Computational Fluid Dynamics on Many-Core Hardware,” AIAA Paper 2015-3050, 2015. doi: Google Scholar

  • [29] Wozniak B. D., Witherden F. D., Russell F. P., Vincent P. E. and Kelly P. H., “GiMMiK—Generating Bespoke Matrix Multiplication Kernels for Accelerators: Application to High-Order Computational Fluid Dynamics,” Computer Physics Communications, Vol. 202, May 2016, pp. 12–22. doi: Google Scholar

  • [30] Vincent P., Witherden F., Vermeire B., Park J. S. and Iyer A., “Towards Green Aviation with Python at Petascale,” Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE Publ., Piscataway, NJ, 2016, p. 1. doi: Google Scholar

  • [31] Wang Z. and et al., “High-Order CFD Methods: Current Status and Perspective,” International Journal for Numerical Methods in Fluids, Vol. 72, No. 8, 2013, pp. 811–845. doi: CrossrefGoogle Scholar

  • [32] van Rees W. M., Leonard A., Pullin D. and Koumoutsakos P., “A Comparison of Vortex and Pseudo-Spectral Methods for the Simulation of Periodic Vortical Flows at High Reynolds Numbers,” Journal of Computational Physics, Vol. 230, No. 8, 2011, pp. 2794–2805.doi: CrossrefGoogle Scholar

  • [33] Rusanov V. V., “Calculation of Interaction of Non-Steady Shock Waves with Obstacles,” Journal of Computational Mathematics and Physics, No. 1, 1961, pp. 267–279. Google Scholar

  • [34] Kennedy C. A., Carpenter M. H. and Lewis R. M., “Low-Storage, Explicit Runge–Kutta Schemes for the Compressible Navier–Stokes Equations,” Applied Numerical Mathematics, Vol. 35, No. 3, 2000, pp. 177–219. doi: CrossrefGoogle Scholar

  • [35] Norberg C., “LDV Measurements in the Near Wake of a Circular Cylinder,” International Journal for Numerical Methods in Fluids, Vol. 28, No. 9, 1998, pp. 1281–1302. doi: Google Scholar

  • [36] Ma X., Karamanos G. S. and Karniadakis G. E., “Dynamics and Low-Dimensionality of a Turbulent Near Wake,” Journal of Fluid Mechanics, Vol. 410, May 2000, pp. 29–65 doi: CrossrefGoogle Scholar

  • [37] Breuer M., “Large Eddy Simulation of the Subcritical Flow Past a Circular Cylinder,” International Journal for Numerical Methods in Fluids, Vol. 28, No. 9, 1998, pp. 1281–1302. doi: CrossrefGoogle Scholar

  • [38] Kravchenko A. G. and Moin P., “Numerical Studies of Flow over a Circular Cylinder at ReD=3900,” Physics of Fluids, Vol. 12, No. 2, 2000, pp. 403–417. doi: CrossrefGoogle Scholar

  • [39] Parnaudeau P., Carlier J., Heitz D. and Lamballais E., “Experimental and Numerical Studies of the Flow over a Circular Cylinder at Reynolds Number 3900,” Physics of Fluids, Vol. 20, No. 8, 2008, Paper 085101. doi: CrossrefGoogle Scholar

  • [40] Williamson C. H. K., “Vortex Dynamics in the Cylinder Wake,” Annual Review of Fluid Mechanics, Vol. 28, No. 1, 1996, pp. 477–539. doi: CrossrefGoogle Scholar

  • [41] Lehmkuhl O., Rodriguez I., Borrell R. and Oliva A., “Low-Frequency Unsteadiness in the Vortex Formation Region of a Circular Cylinder,” Physics of Fluids, Vol. 25, No. 8, 2013, Paper 085109. doi: Google Scholar

  • [42] Jameson A. and Baker T., “Solution of the Euler Equations for Complex Configurations,” AIAA Paper 1983-1929, 1983. doi: LinkGoogle Scholar

  • [43] Witherden F., Park J. and Vincent P., “An Analysis of Solution Point Coordinates for Flux Reconstruction Schemes on Tetrahedral Elements,” Journal of Scientific Computing, Vol. 69, No. 2, Nov. 2016, pp. 905–920. doi: CrossrefGoogle Scholar

  • [44] Witherden F. D. and Vincent P. E., “An Analysis of Solution Point Coordinates for Flux Reconstruction Schemes on Triangular Elements,” Journal of Scientific Computing, Vol. 61, No. 2, 2014, pp. 398–423. doi: CrossrefGoogle Scholar

  • [45] Wood J., Strasizar T. and Hathaway M., “Test Case E/CA-6, Subsonic Turbine Cascade T106,” Test Cases for Computation of Internal Flows, AGARD AR-275, Neuilly-Sur-Seine, France, July 1990. Google Scholar

  • [46] Michálek J., Monaldi M. and Arts T., “Aerodynamic Performance of a Very High Lift Low Pressure Turbine Airfoil (T106C) at Low Reynolds and High Mach Number with Effect of Free Stream Turbulence Intensity,” Journal of Turbomachinery, Vol. 134, No. 6, 2012, Paper 061009. doi: CrossrefGoogle Scholar

  • [47] Pacciani R., Marconcini M., Arnone A. and Bertini F., “An Assessment of the Laminar Kinetic Energy Concept for the Prediction of High-Lift, Low-Reynolds Number Cascade Flows,” Journal of Power and Energy, Vol. 225, No. 7, 2011, pp. 995–1003. CrossrefGoogle Scholar

  • [48] Hillewaert K., de Wiart C. C., Verheylewegen G. and Arts T., “Assessment of a High-Order Discontinuous Galerkin Method for the Direct Numerical Simulation of Transition at Low-Reynolds Number in the T106C High-Lift Low Pressure Turbine Cascade,” ASME Turbo Expo 2014: Turbine Technical Conference and Exposition, American Society of Mechanical Engineers, ASME Paper V02BT39A034, New York, 2014. doi: Google Scholar

  • [49] Garai A., Diosady L. T., Murman S. M. and Madavan N., “DNS of Flow in a Low-Pressure Turbine Cascade with Elevated Inflow Turbulence Using a Discontinuous-Galerkin Spectral-Element Method,” Proceedings of ASME Turbo Expo 2016, ASME Paper GT2016-56700, New York, 2016. Google Scholar

  • [50] Swalwell K. E., “The Effect of Turbulence on Stall of Horizontal Axis Wind Turbines,” Ph.D. Thesis, Monash Univ., Clayton, VIC, Australia, 2005. Google Scholar

  • [51] Haase W., Braza M. and Revell A., DESider–A European Effort on Hybrid RANS-LES Modelling: Results of the European-Union Funded Project, 2004-2007, Vol. 103, Springer Science and Business Media, New York, 2009. doi: Google Scholar

  • [52] Garbaruk A., Leicher S., Mockett C., Spalart P., Strelets M. and Thiele F., “Evaluation of Time Sample and Span Size Effects in DES of Nominally 2D Airfoils Beyond Stall,” Progress in Hybrid RANS-LES Modelling, edited by Peng S. H., Doerffer P. and Haase W., Vol. 111, Notes on Numerical Fluid Mechanics and Multidisciplinary Design, Springer, Berlin, 2010, pp. 87–99. doi: CrossrefGoogle Scholar