38 results about "Partially observable Markov decision process" patented technology

The invention discloses a web service combination method based on depth reinforcement learning for overcoming the problems of long time consumption, poor flexibility and non-ideal combination effect of the traditional service combination method in large-scale service scenes. The depth reinforcement learning technology and the heuristic thought are applied to the service combination problem. In addition, by considering the partial observability of the real environment, the service combination process is converted into a partially-observable Markov decision process POMDP, the solution problem of the POMDP is solved by using a recurrent neural network, and the method still expresses high efficiency encountering the challenge of curse of dimensionality. By adoption of the method provided by the invention, the solution speed can be effectively improved, the dynamic service combination environment is automatically adapted on the basis of ensuring the quality of the service combination scheme, and the adaptability and the flexibility of the service combination efficiency is effectively improved in a large-scale dynamic service combination scene.

Network device selection or handover schemes enable higher network capacity based on partially-observable Markov decision processes. Unavailable cell loading information is observed and/or predicted from non-serving base stations and actions are taken to maintain an active base station set or network device candidate data for selection in routing communications of a mobile device in a mobile device cell selection or handover procedure. A reward function is considered in the selection based on various parameters comprising system capacity, handover times, and mobility of a mobile device or mobile station.

One or more aspects of utility decomposition with deep corrections are described herein. An entity may be detected within an environment through which an autonomous vehicle is travelling. The entity may be associated with a current velocity and a current position. The autonomous vehicle may be associated with a current position and a current velocity. Additionally, the autonomous vehicle may have a target position or desired destination. A Partially Observable Markov Decision Process (POMDP) model may be built based on the current velocities and current positions of different entities and the autonomous vehicle. Utility decomposition may be performed to break tasks or problems down into sub-tasks or sub-problems. A correction term may be generated using multi-fidelity modeling. A driving parameter may be implemented for a component of the autonomous vehicle based on the POMDP model and the correction term to operate the autonomous vehicle autonomously.

The invention relates to an access network service function chain deployment method based on random learning, and belongs to the technical field of wireless communication. The method comprises the steps that an access network service function chain deployment scheme based on partially observable Markov decision process partial perception topology is established for the problem of high delay causedby the physical network topology change under the 5G cloud access network scene. According to the scheme, the underlying physical network topology change is perceived through the heartbeat packet observation mechanism under the 5G access network uplink condition and the complete true topology condition cannot be acquired because of the observation error so that deployment of the service functionchain deployment of the access network slice is adaptively and dynamically adjusted by using partial perception and random learning based on the partially observable Markov decision process and the delay of the slice on the access network side can be optimized. Dynamic deployment is realized by deciding the optimal service function chain deployment mode by partially perceiving the network topologychange based on the partially observable Markov decision process so that the delay can be optimized and the resource utilization rate can be enhanced.

In one embodiment of the invention, a training model for students is provided that models how to present training items to students in a computer based adaptive trainer. The training model receives student performance data and uses the training model to infer underlying student skill levels throughout the training sequence. Some embodiments of the training model also comprise machine learning techniques that allow the training model to adapt to changes in students skills as the student performs on training items presented by the training model. Furthermore, the training model may also be used to inform a training optimization model, or a learning model, in the form of a Partially Observable Markov Decision Process (POMDP).

The invention relates to a frequency spectrum detection method based on a partially observable Markov decision process model. The frequency spectrum detection method comprises the steps that channel state information is added to a channel state historical information sequence, and time delay is estimated so that the channel state information is obtained; the channel initial belief state and state transition probability of each channel are calculated; statistical information of the channel use state and the state transition probability are acquired through observation for a period of time, and a Markov model is established for the use state of each channel; when a time slot increases, the state historical information sequence and the current time slot value are updated; instantaneous remuneration is calculated via combination of the response information update belief state according to the state transition probability of the channels; the value function of each channel after performing different behaviors is calculated; and the maximum discount return acquired by secondary users is calculated and a strategy that the discount total remuneration is the maximum bandwidth is obtained, the channels are ordered in a decreasing way according to the total remuneration of each channel, and a user is guided to try to be accessed to the channels according to the new channel order if data transmission is required.

Disclosed herein is an information-processing apparatus for constructing an information analysis processing configuration to be applied to information analysis processing in an observation domain including an uncertainty. The information-processing apparatus includes a data processing unit for: taking a Partially Observable Markov Decision Process as a basic configuration; taking each of elements included in every information space defined in the Partially Observable Markov Decision Process as a unit; analyzing relations between the elements; and constructing an Factored Partially Observable Markov Decision Process, which is a Partially Observable Markov Decision Process including the relations between the elements, on the basis of results of the analysis.

The invention discloses a method for searching session on the basis of partially observable Markov decision process models. The method includes steps of 1), carrying out preparation phases, to be morespecific, marking corpora on the basis of themes used as units, recording interaction processes when the themes are retrieved by users and generating training files after all retrieval interaction isrecorded; 2), carrying out training phases, to be more specific, initializing statistics, computing initial state transition functions, parsing interactive data of the session from the training filesand updating the state transition functions according to the interactive data; 3), carrying out retrieval phases, to be more specific, receiving initial query statements, and updating the query statements when the users dissatisfy with fed documents until satisfactory document results are fed to the users.

The invention discloses a mammary gland medical record entity recognition labeling enhancement system based on multi-agent reinforcement learning. The system comprises a mammary gland clinical electronic medical record data preprocessing module used for processing original data into a representation form which can be identified and analyzed by a system and analyzing the mammary gland clinical electronic medical record data in terms of medical record contents, structural features, language features and semantic features; a medical clinical entity recognition module used for extracting medical concept entities in texts; and a reinforcement learning labeling enhancement module used for correcting wrong entity labels extracted from mammary gland electronic medical records. According to the method, the multi-agent reinforcement learning model for entity recognition sequence labeling is designed based on the partially observable Markov decision process, the labeling result is corrected, andcompared with a traditional deep learning entity recognition model, the accuracy is effectively improved.

The invention discloses an information interaction method for improving a multi-agent reinforcement learning edge calculation effect. The information interaction method comprises the following steps of constructing an edge calculation communication model based on a partially observable Markov decision process, establishing a shared memory space for executing memory reading, memory filling and memory writing operations on each edge node, setting a target optimization function according to a user cost minimization target and an edge node utility maximization target, setting a time slot length, a time frame length, an initialization time slot and a time frame, obtaining a resource allocation strategy of the edge node, and executing a memory filling operation, enabling the user to execute memory reading and memory writing operations, obtaining a calculation task, a calculation task data volume and calculation capability of each user at the same time, and obtaining a task unloading strategy of the calculation user, optimizing the target optimization function by using a participant-criminator model, and dividing and processing the calculation task. The decision effectiveness of the edge node and the user can be maximized.

The invention discloses a heterogeneous Internet of Vehicles user association method based on multi-agent deep reinforcement learning, and the method comprises the steps: firstly modeling a problem into a partially observable Markov decision process, and then employing the idea of decomposing a team value function, and specifically comprises the steps: building a centralized training distributed execution framework, the team value function is connected with each user value function through summation so as to achieve the purpose of implicit training of the user value functions; and then, referring to experience playback and a target network mechanism, performing action exploration and selection by using an epsilon-greedy strategy, storing historical information by using a recurrent neural network, selecting a Huber loss function to calculate loss and perform gradient descent at the same time, and finally learning the association strategy of the heterogeneous Internet of Vehicles users. Compared with a multi-agent independent deep Q learning algorithm and other traditional algorithms, the method provided by the invention can more effectively improve the energy efficiency and reduce the switching overhead at the same time in a heterogeneous Internet of Vehicles environment.

The invention discloses a computing unloading and resource management method in edge computing based on deep reinforcement learning, which comprises the following steps: constructing an edge computing communication model based on a partially observable Markov decision process, the edge computing communication model comprising M + N agents, the M agents being edge nodes, and the N agents being users; setting a target optimization function according to a user cost minimization target and an edge node utility maximization target; setting a time slot length, a time frame length, an initialization time slot and a time frame; enabling the edge node and the user to respectively use the partial observable Markov decision process to obtain a resource allocation strategy and a task unloading strategy; according to the task unloading strategy and the resource allocation strategy, optimizing a target optimization function by utilizing a participant-criminator model; and dividing and processing the computing task according to the optimized target optimization function. According to the invention, different interest pursues between the edge device and the user are solved, and respective interests are ensured to the maximum extent.

The invention discloses an energy-efficient wireless transmission method for scalable video coding real-time streaming media. The energy-efficient wireless transmission method is characterized by comprising the following steps: implementing selective transmission control on a streaming media scalable coding layer based on channel state estimation with random errors; carrying out performance analysis and strategy optimization based on partially observable Markov decision processes; and adaptively optimizing a transmission control strategy of the coding layer by using an online strategy iteration algorithm. According to the energy-efficient wireless transmission method disclosed by the invention, the optimization of the transmission energy efficiency can be achieved based on adaptive regulation of the traffic through the selective transmission of the video coding layer, and thus the quality QoS of the real-time streaming media video can be ensured, and the transmission energy efficiency can also be effectively increased.

The invention provides an individual movement intervention infectious disease prevention and control method and system. The method comprises the following steps: acquiring daily historical state information and individual relationship information of individual users in a target city within a preset time interval; and inputting the historical state information and the individual relationship information into a trained individual movement intervention infectious disease prevention and control model to obtain prevention and control intervention measures of each user individual in the target city, wherein the trained individual movement intervention infectious disease prevention and control model is obtained by training a graph neural network, a long and short term neural network and an intelligent agent according to sample user individual state information and sample individual relationship information, and the intelligent agent is constructed based on a partial observable Markov decision process; and the individual state information of the sample user comprises health state information of a hidden infected person converted into a dominant infected person. According to the invention, the number of infected people can be reduced as much as possible under low travel intervention.

The disclosure relates to an automatic brain-probe guidance systems. Specifically, the disclosure is directed to a real-time method and system for guiding a probe to the dorsolateral oscillatory region of the subthalamic nucleus in the brain of a subject in need thereof using factored Partially Observable Markov Decision Process (POMDP) via Hidden Markov Model (HMM) represented as a dynamic Bayesian Network (DBN).