61 results about "Sequential decision" patented technology

A system and method for sequential decision-making for customer relationship management includes providing customer data including stimulus-response history data, and automatically generating actionable rules based on the customer data. Further, automatically generating actionable rules may include estimating a value function using reinforcement learning.

Computer software for and a method of determining location of a target image within a two-dimensional input image by employing a three-dimensional reference image comprising determining object edge points in the input image; given estimated seeker line-of-sight orientation and range data, computing a two-dimensional reference image from the three-dimensional reference image; applying a weighted-direction-cross-product Hough Transform to the object edge points and points of the two-dimensional reference image; classifying the input image as containing a target image or not; if the image contains a target image, identifying a location of the target image within the two-dimensional input image; computing confidence measure as to the selected location of the target image; and in the case of sequential decision, also performing confidence accumulation through multi-frames or multi-looks.

This invention introduces a method, system and computer readable media that enables a computer system or computer networked system to perform enterprise portfolio transition optimization. The invention considers several key factors that are traditionally treated informally. The invention provides a sequential decision model for enterprise portfolio transition management of computer and software resources. Using a finite state Markov Decision process, a mathematical formulation is developed to ensure an optimal enterprise portfolio transition plan, with the objective of balancing expected value and risk, that is solved numerically via an approximate dynamic programming algorithm. The output of the model is a set of recommendations in the form of a transition plan for each of the computer and software resources selected within the enterprise portfolio during each phase of a planning horizon.

The invention belongs to the field of maintenance decision-making of hydraulic equipment, and relates to a method for the optimal maintenance decision-making of hydraulic equipment with risk control. The method mainly comprises three steps: 1) judging whether a system is in a status of defect by using a variable-weight association rule algorithm, if so, calculating the probability values of occurrences of latent faults of the system; 2) calculating the comprehensive evaluation value for the consequence of each latent fault by using a BP neural network; and 3) multiplying the probability values obtained in step 1 by the comprehensive evaluation values obtained in step 2 so as to obtain the VaRs (values-at-risk) of the latent faults, judging whether the VaRs are more than a specified threshold, if so, ranking the VaRs in descending order so as to determine the maintenance sequence; otherwise, returning to the step of monitoring. The method can judge whether a device is in a status of defect, judge the type of the latent fault and calculate the probability values of occurrences of latent faults only through a calculation; and compared with traditional risk maintenance methods, the method of the invention improves the accuracy of fault diagnosis, speeds up the diagnosis speed, and provides a better reference for online decision-making.

The invention discloses a knowledge graph inference method based on relation detection and reinforcement learning. The method comprises the steps that on the basis of character string fuzzy matching between a domain knowledge graph and an entity dictionary and a CNN-LSTM-CRF-based entity recognition model, an entity in a question input by a user is detected, and entity detection is completed; relation detection is completed by a neural network based semantic matching model, and the relation detection model is characterized in that low-dimension manifold expression is obtained through the neural network according to the input question, the relation related to the question and the relation not related to the question, on the basis of the low-dimension manifold expression, rank loss optimization model parameters are adopted, so that the question can search the relation set for the relation most similar to the semantics; according to knowledge graph inference based on reinforcement learning, for each time step, on the basis of a strategy function pi theta, under the current entity et, one out-going relation rt+1 is selected, the next entity et+1 is executed, the final entity eT is reached through a preset sequential decision with the maximum inference path length T, and the entity eT is adopted as an answer of the question to be output.

A method for order prioritization includes calculating a cycle time for a product order of a plurality of product orders using an artificial neural network, determining a first order priority of the product order based on a priority index using an analytic hierarchy process, determining a second order priority of the product order based on event based simulation model, and determining a shipping date for the product order based on the second order priority. The artificial neural network calculates the cycle time based upon product order type and a plurality of component counts. The analytic hierarchy process determines a first order priority based upon a plurality of product order attributes. The simulation model determines a second order priority and completion time based upon the first order priority, product model, product type, a plurality of component counts, manufacturing capacity and inventory data, and production time data for historical product orders.

The invention relates to an extra-high-voltage alternating current and direct current transmission mode applicability selecting method and a device using the same. The method includes the steps: firstly, determining index parameters of an extra-high-voltage direct current transmission mode; secondly, establishing alternative schemes for extra-high-voltage transmission modes according to a power transmission scale and establishing a simulation model according to power transmission requirements and conditions of a power grid system; thirdly, calculating the index parameters in the alternative schemes in the second step according to the simulation model; fourthly, determining integrated weight according to the calculated index parameters in the alternative schemes in the third step; and fifthly, performing comprehensive optimization according to the determined index parameters in the alternative schemes in the fourth step based on fuzzy optimization to determine the optical transmission scheme in the alternative schemes. A penalty function is brought into an index calculation process, the requirement of indexes on safety feature can be more effectively met, the problem of multi-objective decision is converted into the problem of sequential decision for a unit system in a multilevel multi-objective level model, the problem of engineering evaluation is theorized, and analysis results are visual and persuasive.

The invention provides a dynamic power system economic dispatch method based on deep reinforcement learning. The method converts dynamic economic dispatch into a multi-stage sequential decision model,takes a dispatch center for action decision as a decision-making main body and an actual power system as an environment, and improves economic dispatch efficiency by designing elements such as actions, states and rewards in reinforcement learning. and converts the economic dispatching model of the power system into a typical multi-stage sequential decision model. According to the model, modelingof an increasingly complex power system is avoided, an accurate thermal power generating unit output cost function is not required, and dynamic economic dispatching of the power system in any scene isrealized through continuous interaction of an intelligent agent and the environment, updating of a strategy, self-adaptive load and uncertainty of new energy output.

Smart in-vehicle decision support system has been developed to address current challenges and offers a new approach to make right stop / go decisions for vehicles to drive through a signalized intersection. The methods and systems described herein exploit a novel conceptualization of the decision support problem as an integration process, where a decision support model takes advantages of vehicle-to-infrastructure communications and fuses the inputs from vehicles and intersection, which comprise key information of vehicle motion, vehicle-driver characteristics, signal phase and timing, intersection geometry and topology, and the definitions of red-light running, to explore a more complete variable space of physical and behavioral information and provide safer and more efficient decision supports to vehicles driving through a signalized intersection than the previous methods and systems. The novel formulation of the decision support model as a probabilistic sequential decision making process incorporates a set of decision rules that are responsible for different situations into the present invention, which enables each decision rule to quickly make a right decision and better improves both traffic safety and intersection throughput than the other existing formulations.

The invention relates to a multi-source remote-sensing image cloud detection method based on self-adaptive thresholds of evidence fusion. Compared with the prior art, the method solves the defect thatit is difficult to carry out cloud detection through combination of multi-data-source fusion and the self-adaptive thresholds. The method includes the following steps: selection and preprocessing ofdata source images; initial-distribution estimation on two types of events of "cloud" and "non-cloud"; classification threshold initial-selection and confidence-interval determination; optimized classification based on order decision trees and evidence fusion; and output of cloud detection classification results. According to the method, a manner of combination work of a multi-wave-band optical sensor and a multi-angle polarization detector is adopted, multi-data-source data needing to be used in "cloud" detection are fused, the accuracy problem of "cloud" and "non-cloud" classification is effectively solved, and classification precision in cloud edge detection is improved.

The invention provides a cost-sensitive incremental face recognition method based on information entropy selection. The cost-sensitive incremental face recognition method is composed of a deep convolutional neural network part, an information entropy-based sample selection part and a cost-sensitive sequential three-branch decision classification part. The information entropy is used for assessingthe information amount of the classification result of the face recognition sample, so that the system can automatically assess the unlabeled sample information amount, and samples with large information amount are selected for manual labeling; a face recognition problem is regarded as a sequential process of information granularity from coarse to fine by utilizing the thought of three sequentialdecisions sensitive to cost, each iterative loop added with a marked sample is used as a decision step of the three sequential decisions, and the minimum cost recognition effect of the sample in eachdecision step is given according to the minimum Bayesian risk principle.

The embodiment of the invention discloses a method and a device for allocating resources for a wireless terminal directly connected user (D2D). The method comprises the following steps of: formulatinga reinforcement learning (RL) algorithm according to the uncertainty of a wireless channel state and a state transition possibility. Each pair of D2D users can be viewed as an independent agent, making decisions based on the channel state it observes locally. In order to solve the problem of sequential decision-making in multi-user systems, a multi-agent RL algorithm is proposed. We assume that D2D users do not have any information about the availability and quality of the resource blocks to be allocated, so the problem is modeled as a stochastic non-cooperative game. Therefore, each agent becomes a participant in the non-cooperative game, and all participants make decisions together in order to achieve the optimization of all benefits. The embodiment of the invention can lighten the loadand improve the capacity of the cellular network, and the system throughput can be maximized through effective energy management.

The invention discloses a user real-time autonomous energy management optimization method based on near-end strategy optimization. The management optimization method comprises the steps of S1, classifying and modeling user DER equipment; S2, modeling a user real-time autonomous energy management optimization problem into a sequential decision problem based on classification and modeling of the user DER equipment in the step S1; S3, extracting the future trend of real-time time sequence data by using a long short-term memory neural network, and assisting the deep reinforcement learning in the steps S4 and S5 to carry out strategy optimization; S4, inputting the future trend extracted in the S3 and internal state characteristics observed by an energy management intelligent agent into a strategy function based on a deep neural network, enabling the energy management intelligent agent to learn discrete and continuous actions at the same time, and achieving the control of the equipment; and S5, learning an energy management optimization strategy in the discrete and continuous actions in the step S4 by adopting a near-end strategy optimization algorithm. According to the management optimization method, the adaptability of the strategy to uncertainty is improved while the power consumption cost is minimized.

The invention discloses an efficient routing method for an orthogonal circular orbit configuration satellite constellation. According to the invention, the method comprises the steps: proposing a principle of proximity, a principle of approach and an equatorial orbit satellite active periodic updating mechanism according to a constellation link model to ensure the rationality and the cheapness ofa network link; proposing a link priority algorithm based on hop count prediction on the basis of the link model; enabling a satellite to send out a data packet according to the priority of each linkthrough sequential decisions of a link actual on-off state, a link pseudo-off state caused by a non-backhaul strategy, a destination node direction, a link congestion degree and a link propagation delay factor; carrying out discussion in four situations according to the categories of the source satellite node and the destination satellite node, and providing a hop count prediction method for sending data packets from different links for each situation according to satellite numbers and state information.

The invention relates to an optimal power flow calculation method considering discrete and sequential decision variables for a large-scale power distribution network. The optimal power flow calculation method is characterized by comprising the following steps of building a mathematical model of a complicated three-phase unbalanced power distribution network, carrying out complicated optimal power flow calculation decomposition, and carrying out a simplification process on network loss calculation of the three-phase unbalance power distribution network. By the optimal power flow calculation method, the complicated optimization problem can be decomposed into an integral linear programming primal problem and a non-linear feasible subproblem, decision making optimization and system security constraint are organically combined by using mutually-iterated calculation decisions, the optimal power flow is embedded to a decision process, and meanwhile, rapid calculation on asymmetric power flows at a large scale is achieved. The optimal power flow calculation method is suitably used for three-phase unbalance power flow calculation, looped network structure, bidirectional power flows, real-time decision of discrete and sequential decision and optimal power flow, and has the advantages of scientific reasonableness, high adaptability, operation accuracy, high speed and the like.

The invention provides a multichannel transmission scheduling method based on time domain interference alignment in an underwater acoustic network. The method comprises the following steps: S1, network topology representation is carried out, a time slot division model is adopted, the whole message transmission process is divided into a plurality of time slots with the length of [tau], the transmission delay between nodes refers to the time required by message transmission between two nodes, a transmission delay matrix is used for representing a network topology structure, and elements in the matrix represent the number of the time slots required by message transmission between the nodes; S2, transmission scheduling is initialized, the states of the nodes are stored through a three-dimensional matrix, and transmission time slots of the nodes, selected transmission channels and destination nodes are determined; and S3, optimal transmission decision search is carried out, an optimal scheduling problem is searched to be regarded as a sequential decision problem, and the problem can be solved by using dynamic planning. In the multi-channel network model, a plurality of nodes can transmit messages in one time slot at the same time, so that data conflicts are reduced, and the network throughput is increased.

The invention belongs to the technical field of production plans, and particularly relates to a single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning, which comprises the following steps of: (1) modeling a job shop scheduling environment by adopting a disjunction graph method, converting a scheduling decision problem into a sequential decision problem, establishing a Markov quintuple model, solving the model by using deep reinforcement learning; (2) extracting a current state from a disjunction graph environment scheduled by the job shop; (3) fitting the action value function and the target value function by adopting a convolutional neural network; (4) adopting 18 heuristic scheduling rules as proxy actions of reinforcement learning; (5) designing a reward function to evaluate the whole scheduling decision, and updating a weight parameter of an action value function by using a DQN algorithm; (6) performing state transition; and (7) updating network parameters of the target value function; the scheduling problem of the job shop can be quickly processed, and the method has the advantages of being high in real-time performance and high in flexibility.

A method and an apparatus accurately discriminates between speech and voice-band data (VBD) in a communication network by calculating self similarity ratio (SSR) values, which indicate periodicity characteristics of an input signal segment, and / or autocorrelation coefficients, which indicate spectral characteristics of an input signal segment, to generate a speech / VBD discrimination result. In one implementation, the speech-VBD discriminating apparatus calculates both short-term delay and long-term delay SSR values to analyze the repetition rate of an input signal frame, thereby indicating whether the input signal frame has the periodicity characteristics of a typical speech signal or a VBD signal. The speech-VBD discriminating apparatus further calculates a plurality of short-term autocorrelation coefficients to determine the spectral envelope of an input frame, thereby facilitating accurate speech / VBD discrimination. According to one implementation of the present invention, the speech-VBD discriminating apparatus relies on sequential decision logic which improves classification performance by recognizing that changes from speech to VBD or vice versa in a communication medium are unlikely, and discounts discrimination results for relatively low-power signal portions which are more susceptible to errors to further improve discrimination accuracy.

A system and method for defining and calibrating the inputs to a sequential decision problem using historical data, where the user provides historical data and the system and method forms the historical data (along with other inputs) into at least one of the states, actions, rewards or transitions used in composing and solving the sequential decision problem.

The invention relates to the technical field of brain-computer interfaces, especially a parameter optimization method for a decision-making model of a brain-computer interface system. The method comprises the following steps: (1), collecting brain-computer training data, and carrying out the preprocessing; (2), carrying out data classification through a linear space integrated single detection method, and the positioning of an evidence-accumulating start process. The method employs the linear space integrated single detection method to carry out the classification and recognition of two experiment conditions in training data, carries out the time locating of the evidence-accumulating process in a decision-making model through recognizing the change tendency of the accuracy with time in a single experiment, and carries out the parameter optimization of a sequential decision model. Compared with a previous sequential decision model, the method carries out the locating of the evidence-accumulating process in the decision-making model, eliminates the ineffective classified information accumulation process, carries out the accumulation of the effective classification information in the evidence-accumulating process, and improves the instantaneity of the brain-computer interface system based on the decision-making model.

An adaptive signal detector to detect a signal in a spectrum space includes a software defined receiver for calculating a cumulative decision statistic for determining the presence of the signal in the spectrum space. The cumulative decision statistic is determined by a sequential energy detector that includes a block energy detector that has a block termination rule, a block sampling instant and a block decision phase. A truncated sequential energy detector has a truncated sequential termination rule and a truncated sequential decision phase. The block energy detector and truncated sequential energy detector are interoperated to provide the cumulative decision statistic in which the block energy sampling instants provide samples that are independent and identically distributed. The cumulative decision statistic determines the presence of the signal in the spectrum space based upon a non-Bayesian dynamic mathematical modeled relationship of the cumulative decision statistic.

A method for order prioritization includes calculating a cycle time for a product order of a plurality of product orders using an artificial neural network, determining a first order priority of the product order based on a priority index using an analytic hierarchy process, determining a second order priority of the product order based on event based simulation model, and determining a shipping date for the product order based on the second order priority. The artificial neural network calculates the cycle time based upon product order type and a plurality of component counts. The analytic hierarchy process determines a first order priority based upon a plurality of product order attributes. The simulation model determines a second order priority and completion time based upon the first order priority, product model, product type, a plurality of component counts, manufacturing capacity and inventory data, and production time data for historical product orders.

The invention provides a power system multi-stage robust unit commitment decision-making method considering the source-load dual uncertainty, wherein a power system multi-stage robust unit commitment model considering wind power and load uncertainty is established. The method comprises the following steps: firstly, aiming at uncertainty of wind power output and power load, constructing a high-dimensional uncertainty set based on upper and lower boundaries of the wind power output and the power load; then, a multi-stage robust unit combination model based on the high-dimensional uncertainty set is established in combination with sequential decision characteristics of power system dispatching; and finally, in combination with a linear affine rule and robust peer-to-peer transformation, converting the robust optimization problem in the min-max form into a mixed integer linear programming problem for solving. The method can be applied to unit combination decision making of a wind power-containing electric power system, and is beneficial to improving the reliability and the effectiveness of an electric power system scheduling decision under an uncertain operation condition.

The invention provides an aero-engine remaining service life estimation method based on deep reinforcement learning, and the method comprises the steps: taking the requirements and characteristics ofaero-engine life estimation as a sequential decision problem, and establishing a corresponding Markov decision process model; for the Markov model, formulating interaction rules such as state action rewarding and the like, designing a life estimation rewarding function, and providing a deep learning model of a life estimation strategy according to the characteristics of aero-engine life estimationdata; and finally, learning an optimal aero-engine life estimation strategy in the Markov model by using a deep reinforcement learning algorithm. According to the method, the over-fitting risk brought by a conventional supervised learning method is overcome, a better aero-engine service life estimation strategy can be obtained, the accuracy of aero-engine service life estimation can be improved,the timeliness of maintenance according to conditions is improved, the safety of aircraft flight is enhanced, and unnecessary maintenance cost is saved.

The invention relates to the field of big data processing, and discloses a diet plan decision-making method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining current physiological state data and target physiological state data of a user; taking the current physiological state data and the target physiological state data as input of a sequential decision model of reinforcement learning, and outputting a plurality of execution plans and reward values of the execution plans based on the sequential decision model of reinforcement learning; and selecting an execution plan meeting a preset condition as a target execution plan according to the reward value, and outputting the execution action and physiological state data of each day included in the target execution plan. According to the invention, the diet data of the long-term plan can be output, and the planning efficiency of the diet plan is improved.

The invention discloses a container-based edge computing resource allocation method and system for the Industrial Internet of Things. The method is as follows: according to the data types collected by the perception layer, the tasks are divided into n types, and through historical data statistics, it is obtained that the tasks arrive at the edge successively Probability relationship on the server cluster; according to the system state space and decision space, allocate computing resources to real-time tasks and make sequential decisions; according to the resource allocation algorithm based on reinforcement learning, the system allocates resources from low, medium and high according to the current state Select the most reasonable strategy among the strategies to allocate resources to the current task; build a task scheduling processing model, create a container, and then schedule data to the container for processing and analysis. After the task is completed, delete the container to complete the allocation. In the case of limited resources, the present invention allocates different types of data computing resources, creates containers for processing and analysis, and improves resource utilization efficiency and the total rate of task processing. The invention is applicable to the field of data processing of the industrial internet of things.

A system and method for optimizing management of a network by transforming the network into some or all of the elements of a sequential decision problem, then composing a sequential decision problem; error and convergence checking the sequential decision problem; and solving the sequential decision problem. The solution to the sequential decision problem is provided to a user as decision advice or automatically implemented. The network may be a website designed for access by users of mobile or other devices, or a logistics network for the distribution of goods, or a utility network for communications, energy, roads or other services. The decision advice allows the user to optimize the network to improve an aspect of the network, such as the amount of time spent traveling through the network (for roads) or the total-value derived from visitors (for a website selling products).