# Guide Practical Optimization: Algorithms and Engineering Applications

The iterative algorithm applies at each iteration a search directed to the steepest descent. For this, obviously, gradient information is needed. Due to the lack of a step size, a line search is applied at each iteration step. Often the norm of the gradient is used a first-order termination condition for a steepest descent search. As in most cases, the optimal search direction pointing to the minimum differs largely from the gradient direction, the steepest descent search results in a highly inefficient, but typical zigzag pattern. Consequently, the method of steepest descent has found little applicability in practice, since its performance is in most cases largely inferior to second-order methods, like the Quasi-Newton method.

For high-dimensional design spaces, though, the method of steepest descent can be advantageous, since it does not require the estimation of the inverse Hessian, which scales with the number of design variables.

The set-up and maintenance of a steepest descent method is not simple, but also not too difficult. Special care has to be taken for the accuracy of the gradient estimation. Besides, parameters of the line search and termination condition have to be set properly, and have to be monitored as the optimization proceeds. Using the method of steepest descent based on gradients calculated using the adjoint approach in combination with the free shape parametrization has shown to be impractical, as can be seen in Section 4. Thus, a smoothing is applied to the gradients, where the gradient of each design parameter is smoothed using weighted gradients of the neighbouring design parameters.

Especially in areas of large flow parameter gradients, e. Still, a weighted smoothing influences the gradient direction and therefore is not a pure steepest descent method any more. All members of the family of Quasi-Newton methods are second-order methods, which require a continuous and two-times differentiable goal function.

In Quasi-Newton methods, the second-order information Hessian is not computed directly. To build up proper information inside the approximated Hessian, typically N iterations are needed in an N -dimensional design space.

Machine Learning and Robust Optimization, Fengqi You, Cornell University

This is one of the weaknesses of Quasi-Newton methods, which makes it unstable and inefficient for a large number of design variables. The typical termination conditions for Quasi-Newton algorithms is the norm of the gradient. On the basis of the Hessian, Quasi-Newton methods can derive proper search directions pointing to the minimum and to the steepest descent , with a proper step length.

## Practical Optimization: Algorithms and Engineering Applications / Edition 1

As such, the methods are highly efficient and inefficient zigzag search paths are avoided. Some codes use first-order forward differences, but here second-order central differences are used. Beside the approximation and neglect of higher order terms, a principal problem of finite differences is the evaluation of the step size h , which determines the magnitude of the change of the design parameters. A too small step size can lead to round-off errors whereas a too big step size can result in truncation errors.

An improper choice of the step size can thus lead to erroneous gradients and poor optimization results. In practice, before using finite differences, a sound gradient and noise study is required to identify proper step sizes for each design variable. If step sizes are chosen properly and noise in the goal function is not too large, FD is a reliable and stable gradient estimator. The computational cost for FD is comparably high, since the number of goal function evaluations scales linearly with the number of design variables.

With this, in the realm of shape optimization in fluid dynamics, FD requires tremendous computational resources, since a the number of design variables is high, and b the computational cost for a single evaluation of the goal function is high. In this paper, the symmetrical FD scheme is used, where proper step sizes had been chosen in each case, with the help of a gradient study.

By this the high computational cost of FD for a large number of design variables can be avoided, since the computational effort is independent of the number of design variables. But another set of partial differential equations—the adjoint equations—have to implemented and solved numerically. In practice, the adjoint approach for gradient estimation imposes several additional tasks on the CFD practitioner, such as implementation of the adjoint equations and maintenance of the convergence behaviour of a second set of field equations.

Besides, the continuous adjoint equations often impose simplifications, i. As such, both is benchmarked here, gradient estimation via FD and via the solution of adjoint equations. In this paper, the continuous adjoint approach is used, since OpenFOAM is highly suitable for the implementation of continuous equations.

The implementation of the adjoint field follows the available adjoint solver for ducted flows by Othmer, de Villier and Weller Therein the adjoint turbulent viscosity is supposed to be equal to the turbulent viscosity of the flow field. This leads to high quality requirements regarding the numerical grid and a careful case set-up.

In this work, the flow field is pre-converged, before the adjoint field is solved, which leads to a stable and converging adjoint field. The iterations needed for pre-convergence are added to the total amount of iterations needed for the optimization case. The gradient information can be used directly for pure free shape parametrization, or treated further, e.

The adjoint method computes gradients on the whole geometry surface, which can also be interpreted as production sensitivities. The designer can use the size of the sensitivities to define the importance and range of production tolerances. In the process, large sensitivities indicate where tolerances should be fulfilled and low tolerances where can be higher.

## Practical optimization. Algorithms and engineering applications

By this, the production costs can be controlled precisely. In order to show the potential and drawback of free shape deformation, a preliminary study on optimizing an airfoil using the adjoint approach and steepest descent for all surface points as design parameters is conducted. Figure 2 shows the development of the lift coefficient during this optimization.

The goal function is a quadratic least square function as shown in Equation 1 , without any penalty functions. In this example, all surface points are free to move and the gradients are neither smoothed nor transformed in any way. As mentioned before, this is necessary, since the adjoint approach as it is implemented here does not support discontinuities. The resulting shapes of this optimization are shown in Figure 3 , where a close-up view of leading and trailing edge are shown.

Since the adjoint field and therefore also the gradients are driven by the flow field, relatively large geometry changes are expected at areas with large changes in the flow field. The overall changes in the geometry are small and thus only close-up views of the leading and trailing edge are presented. This bump may not be useful for a practical application of this airfoil, but it is mathematically correct, since the goal lift coefficient is reached.

The tendency of developing bumps can lead to issues with convergence of the primal and adjoint field. Hence, a free shape parametrization can only be used, if changes in the goal function and thus the shape deformations are small. This was the case for the presented example, but will not hold for the following optimizations. In order to overcome such a problem additional smoothing approaches can be used, e.

Also a projection of gradients to a lower design space can be applied to overcome infeasible geometries, which is presented in the following Section 4. For free shape deformation in this work, a simple smoothing of the gradients is applied by weighted averaging over neighbouring surface points. This results in a reduction of the effective design variables, but still each airfoil point is moved individually.

Beside the use of the adjoint approach for optimization, a practical application is the use of the gradient information for sensitivity maps. The designer can manually include the sensitivities of a given objective function in the design process of a geometry. This is rather a cumbersome improvement process instead of an optimization, but can be done in cases, when too many constraints have to be considered, e.

The strength of the adjoint approach is that the gradient computation does not scale with the number of design parameters and thus each airfoil point can be used as an individual design variable.

3. Find a copy in the library!

In practice this is rarely done, since high gradients in the primal flow lead to strong adjoint gradients in the affected regions, resulting in uneven deformations such as bumps. These can be mathematically correct results of an optimization, but they are seldom suitable for standard engineering applications, which can be seen in Figure 3 , where the large flow acceleration along the leading edge leads to large gradients in some regions and therefore to large deformations.

The grid on the surface of the airfoil contains surface points and the adjoint method leads to gradient information on each of these points. A reduction to very few design variables might not be useful in combination with the adjoint approach, as the solution of the adjoint equations needs some additional computational effort, which may not pay off, e.

However, it is done in this work in order to compare the different optimization algorithms and to evaluate when the adjoint approach increases the speed of the optimization and when standard finite-differencing is sufficient for the gradient computation. Besides, a projection of the gradients to lower dimensions also applies an indirect smoothing. The chain rule is used for the gradients of the goal function f x s , y s within the two-dimensional coordinate system where the surface points are defined in. The x - and y -coordinates depend on the selected parametrization parameter s m of the chosen parametrization, e.

This projection of gradients is exact, since for each design parameter the gradients along the whole surface are taken into account. It can easily be extended for 3D applications by adding the z -direction. Different parametrizations are used within different optimization methods, but the general CFD set-up of the simulations is the same in order to ensure a valuable comparison. The numerical grid as it is used here is a compromise between the different optimization methods in this work. The Nelder—Mead algorithm is generally able to handle a higher amount of noise in the objective function than a Quasi-Newton method.

The adjoints are driven by the primal flow field and any numerical error is directly transferred to the adjoint field, leading to noise in the gradient evaluation. Thus a fine grid is needed in the adjoint approach, but a coarse grid could be used with the Nelder—Mead algorithm and a standard RANS simulation of an airfoil could probably also deliver adequate results with a coarse grid.

For practical applications, the computational accuracy has to be high in order to find improvements over airfoils, which are commonly in use. In principle, the simple airfoils in this work could also be optimized with a smaller accuracy, when using Nelder—Mead algorithm.

• 50 Easy Weekend Scroll Saw Projects!
• CGS SALES & SERVICE;
• Family relationships and familial responses to health issues.
• The Complete Idiots Guide to European History;
• Kundrecensioner!
• But for the gradient computation via adjoints or finite differences , a reasonably high accuracy is necessary. This necessity is the minimum limitation for the computational costs and any higher accuracy is not computed in order to limit the costs. After some preliminary testing, the grids are created as described in the following.