An efficient gradient forcasting search method utilizing the discrete difference equation prediction model

Abstract

Optimization theory and method are very important in numerous different domains of engineering design and applications. Compared to many previously proposed search methods, the gradient descent method is simple and widely employed to solve numerous different optimization problems. However, the gradient descent method is easily trapped into a local minimum and converges slowly. A Gradient forecasting search method (GFSM) for improving the performance of the gradient descent method for resolving optimization problems is proposed herein. The GFSM is based on the gradient descent method and on the universal Discrete Difference Equation Prediction Model (DDEPM) proposed herein. The concept of the universal DDEPM is derived from the grey prediction model. The original grey prediction model employs mathematical hypothesis and approximation to transform a continuous differential equation into a discrete difference equation. This is not a logical approach because the forecasting sequence data is invariably of discrete type. To construct a more precise prediction model.this work adopts a discrete difference equation. The GFSM proposed herein can accurately predict the precise searching direction and trend of the gradient descent method via the universal DDEPM and can adjust prediction steps dynamically using the golden section search algorithm. Experimental results indicate that the proposed method can accelerate the searching speed of gradient descent method and can help the gradient descent method escape from local minima.Our results further demonstrate that applying the golden section search method to achieve dynamic prediction steps of the DDEPM is an efficient approach for this search algorithm.