Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Information physical system safety control method based on deep reinforcement learning

A cyber-physical system and reinforcement learning technology, which is applied in the field of security control of cyber-physical systems based on deep reinforcement learning, can solve problems such as poor control performance of security control strategies, and achieve improved control performance, strong robustness, and guaranteed stability Effect

Active Publication Date: 2022-01-04
HARBIN INST OF TECH
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the problem of poor control performance of security control strategies designed based on existing methods in the case of network attacks, and propose a cyber-physical system security control method based on deep reinforcement learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Information physical system safety control method based on deep reinforcement learning
  • Information physical system safety control method based on deep reinforcement learning
  • Information physical system safety control method based on deep reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach 1

[0071] Specific implementation mode 1. Combination figure 1 This embodiment will be described. A cyber-physical system security control method based on deep reinforcement learning described in this embodiment, the method specifically includes the following steps:

[0072] Step 1. Establish a cyber-physical system model under the false data injection attack of the actuator;

[0073] Step 2. Describe the cyber-physical system model under the false data injection attack of the actuator established in step 1 as a Markov decision process;

[0074] Step 3: Build a deep neural network, and output a decision strategy for the Markov decision process based on the built deep neural network.

specific Embodiment approach 2

[0075] Specific implementation mode two: the difference between this implementation mode and specific implementation mode one is that the specific process of the step one is:

[0076] Step 11. Ideally, the dynamic equation of the cyber-physical system model is:

[0077]

[0078] in, represents the state vector of the cyber-physical system, represents the field of real numbers, n x Indicates the dimension of the state vector x, Indicates the control signal to be designed, n u Indicates the dimension of the control signal u, is the first-order derivative of x, and f(·) represents the generalized function mapping;

[0079] Step 12. Discretize the cyber-physical system model in step 11 based on the Euler method to obtain a discretized cyber-physical system model:

[0080] x(k+1)=(f(x(k),u(k)))Δt+x(k)

[0081] Among them, x(k) represents the state vector of the discretized cyber-physical system at time k, u(k) represents the control signal at time k, Δt represents the...

specific Embodiment approach 3

[0092] Embodiment 3: This embodiment is different from Embodiment 1 or 2 in that the attack distribution matrix Γ is a diagonal matrix, and the values ​​of the diagonal elements are all 0 or 1. If the i-th actuator is attacked , then the i-th diagonal element of the attack distribution matrix Γ (that is, the element in the i-th row and i-column of the diagonal matrix, corresponding to the i-th actuator) takes the value 1, otherwise, the i-th diagonal element takes the value 0 .

[0093] Other steps and parameters are the same as those in Embodiment 1 or Embodiment 2.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an information physical system security control method based on deep reinforcement learning, and belongs to the technical field of information security. According to the invention, the problem of poor control performance of a security control strategy designed based on an existing method under the condition of network attack is solved. According to the method, the dynamic equation of the cyber-physical system under the attacked condition is described as a Markov decision process, and based on the established Markov process, the security control problem of the cyber-physical system under the false data injection attack condition is converted into a control strategy learning problem only using data; based on a flexible action-critic reinforcement learning algorithm framework, a flexible action-critic reinforcement learning algorithm based on a Lyapunov function is proposed, a novel deep neural network training framework is provided, a Lyapunov stability theory is fused in the design process, the stability of an information physical system is ensured, and the control performance is effectively improved. The method can be applied to safety control of the information physical system.

Description

technical field [0001] The invention belongs to the technical field of information security, and in particular relates to a cyber-physical system security control method based on deep reinforcement learning. Background technique [0002] The cyber-physical system integrates and develops existing communication, wireless network, distributed, artificial intelligence and other technologies, and builds the mutual mapping, timely interaction, and efficient Collaboration will become a new generation of intelligent system integrating computing, communication and control. As the core of intelligent manufacturing, the development of cyber-physical systems has been valued by countries all over the world. In recent years, frequent cyber attacks have posed a great threat to national security, economic development, infrastructure security, and people's lives and property. This makes how to ensure the security of cyber-physical systems a major problem that needs to be solved urgently. A...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G05B13/04
CPCG05B13/042
Inventor 吴承伟柴庆杰刘健行孙光辉吴立刚
Owner HARBIN INST OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com