DBN model optimization method based on a PSO algorithm

Abstract

The invention provides a DBN model optimization method based on a PSO algorithm. The DBNs is a probability generation model. The method is opposite to a neural network of a traditional discriminationmodel. The generation model is used for establishing joint distribution between observation data and labels; According to the invention, both P (Object | Label) and P (Label Object) are evaluated, andthe discrimination model only evaluates the P (Label Object). Wherein the DBNs is composed of a plurality of Restricted Boltzmann Machanes layers, and the type of a typical neural network is shown inthe figure. These networks are restricted into a visual layer and a hidden layer, there being connections between the layers, but there is no connections between the units within the layers. Hidden layer units are trained to capture correlation of high-order data represented at a visual layer. In order to make prediction of the DBN model more accurate, optimization needs to be conducted on the DBN, and the optimization comprises the number of layers, the number of units of each layer, the learning rate and the like. Therefore, the invention provides a DBN model optimization method based on aPSO algorithm. the optimal unit number and the RBM optimal learning rate of the input layer and the hidden layer are searcjed by adopting a parallel PSO (Particle Swarm Optimization) algorithm, and parallelization on a Spark platform is carried out based on a MapReduce principle.

Description

technical field [0001] The invention relates to deep learning, model optimization and intelligent prediction, in particular to a method for optimizing a DBN model based on a PSO algorithm. Background technique [0002] Particle swarm optimization simulates birds in a flock of birds by designing a massless particle. The particle has only two attributes: speed and position. The speed represents the speed of movement, and the position represents the direction of movement. Each particle searches for the optimal solution independently in the search space, and records it as the current individual extremum, and shares the individual extremum with other particles in the entire particle swarm, and finds the optimal individual extremum as the entire particle The current global optimal solution of the swarm. All particles in the particle swarm adjust their speed and position according to the current individual extremum found by themselves and the current global optimal solution shared ...

AI Technical Summary

Problems solved by technology

Its advantage is that because it calculates the loss of a single training sample, the amount of calculation is small; its disadvantages include: because only the gradient of a single sample is considered, it is easy to fall into a local optimum; due to the introduction of noise, it has regularization Effect; it is sensitive to parameters, so parameter initialization is very important; when the sample set is large, the training time is long; it is difficult to choose an appropriate learning rate

Images