Qualification trace calculator based on phase change memory

Abstract

The invention discloses a qualification trace calculator based on a phase change memory. The qualification trace calculator comprises a phase change memory array and a result converter. The multi-valued characteristic of the phase change memory is utilized, qualification trace data is stored in the memory units in a conductive mode, and compared with a traditional binary storage mode, the number of the memory units can be effectively reduced, and high-density storage is achieved; attenuation operation along with time is spontaneously realized by utilizing the conductance drift effect of the phase change memory, other operational circuits are not needed, and the hardware overhead of operation is effectively reduced; and storage and attenuation operation of qualification trace data are carried out in the phase change memory, so that huge energy consumption caused by frequent data carrying is avoided. In addition, by adjusting parameters in the result converter, the attenuation speed of the qualification trace can be flexibly adjusted, so that the method is suitable for reinforcement learning tasks with different requirements. The limitation of a storage wall in a traditional calculation architecture can be broken through, and further development of reinforcement learning is promoted.

Description

technical field [0001] The invention belongs to the field of new computing technologies, and in particular relates to a qualification trace calculator based on the inherent conductance drift effect of a phase change memory. Background technique [0002] Due to its excellent performance, the reinforcement learning algorithm has attracted the attention of many researchers in recent years. It can generate a problem-solving strategy based on the rewards and punishments of the task environment. The optimization strategy generated through multiple iterations can effectively complete complex tasks in many fields. , and does not require guidance or supervision from outside. The continuously optimized reinforcement learning algorithm has been able to achieve performance close to or even surpassing human level in areas such as autonomous driving and game battles. The powerful function of the reinforcement learning algorithm is inseparable from an effective and commonly used mechanism...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the problem of too high energy consumption of entitlement trace calculation in complex reinforcement learning algorithms, the present invention provides an entitlement trace calculator based on phase change memory multi-valued characteristics and conductance drift characteristics, which can be realized spontaneously in the way of in-memory calculation The attenuation of eligibility traces, thus greatly reducing the energy consumption of eligibility trace calculations

Images