A PSO-BFGS neural network training algorithm

Abstract

The invention discloses a PSO-BFGS neural network training algorithm. The method comprises the following steps: (1) dividing tasks; (2) utilizing the PSO algorithm for global search, using the generated optimal solution as the input of the BFGS by iteration, carrying out local fine search through the BFGS algorithm, (3) carrying out parallelism degree division on the PSO algorithm and the BFGS quasi-Newton algorithm, (4) achieving a neural network training error evaluation function through the PSO algorithm and the BFGS algorithm, (5) achieving the PSO algorithm, The algorithm is a neural network training parallel algorithm based on a particle swarm optimization (PSO) and a BFGS quasi-Newton algorithm, a GPU is adopted as a neural network training computing device, and compared with otheroptimization algorithms, the algorithm has high convergence efficiency and global search capability. OpenCL is used as a programming language for realization, and compared with the realization of using a CUDA language, the realization of using OpenCL as the programming language has higher transportability and can be used on GPUs and FPGAs of different manufacturers.

Description

technical field [0001] The invention belongs to the field of high-performance computing and machine learning, and specifically relates to a PSO-BFGS neural network training algorithm. Background technique [0002] Artificial neural network is an information processing system, it can learn any input-output relationship through a series of data, and establish an accurate model. Currently, one of the main challenges facing artificial neural networks is training. Before training, the neural network does not carry any information; after training, the weight value of the neural network can be determined, so as to establish an accurate model based on the training data. The process of determining the weight value of the neural network is an optimization process, that is, the neural network obtains a more accurate fitting weight value through repeated iterative calculations through various optimization algorithms. Traditional neural network training optimization algorithms are main...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a kind of PSO-BFGS neural network training algorithm in order to overcome the deficiencies in the prior art, namely based on the neural network training parallel algorithm of particle swarm algorithm (PSO) and BFGS quasi-Newton algorithm, adopt GPU as neural network Network training computing equipment, limited to solve the problem of local convergence in the training process of traditional artificial neural networks while taking into account the convergence speed

Images