Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A method and system for judging the reliability of distributed clusters through weighted entropy

A distributed cluster and weighted entropy technology, applied in instrumentation, error detection/correction, computing, etc., can solve problems such as the difficulty of cluster system fault location and troubleshooting, the impact of job execution speed, and the large system uncertainty

Active Publication Date: 2019-11-15
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0012] When the inventor conducted research in a cluster environment, he found that the performance of the same type of job would fluctuate when running on the same cluster, that is, the execution speed of the job would be affected by the cluster environment
In the cluster environment, due to the uncertainty of the fault, it will be difficult to locate and eliminate the fault of the cluster system, and the maintenance cost will be high; and because the system entropy is the measure of uncertainty in the computer system, the higher the system entropy, the more uncertain the system will be. more sex

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and system for judging the reliability of distributed clusters through weighted entropy
  • A method and system for judging the reliability of distributed clusters through weighted entropy
  • A method and system for judging the reliability of distributed clusters through weighted entropy

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The general idea of ​​the present invention is that, firstly, the log events of different severities (the log events selected here are log events indicating system errors or failures) are respectively assigned weights in advance, and then the sliding window is used to count the The number of different types of system logs, and then calculate the error rate of each node, and calculate the reliability weighted entropy of the cluster according to the error rate, and finally judge the stability of the cluster according to the fluctuation.

[0049] In order to make the above-mentioned features and effects of the present invention more clear and understandable, the following specific examples are given together with the accompanying drawings for detailed description as follows.

[0050] Such as figure 1 As shown, the present invention proposes a method and system for judging cluster reliability based on weighted entropy. The method includes the following steps:

[0051] Step...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method and a system for judging the reliability of a distributed cluster through a weighted entropy, and belongs to the field of distributed cluster reliability. The methodcomprises the steps of previously performing weighting for log events with different severities; then performing statistics on quantities of different types of system logs of nodes of the cluster in atime period by utilizing a sliding window; calculating error rates of the nodes; according to the error rates, calculating the reliability weighted entropy of the cluster; and finally according to afluctuation range, judging the stability of the cluster. Through the method, system reliability is quantized; the reliability of the whole cluster not only can be quantitatively judged through simpledeployment but also can be monitored in real time to obtain environmental change of the cluster in time; and when the reliability of the cluster has a relatively great fluctuation, corresponding measures can be adopted in time for processing.

Description

technical field [0001] The invention relates to the field of distributed cluster reliability, in particular to a method and system for judging the reliability of distributed clusters through weighted entropy. Background technique [0002] Cluster technology, as the name suggests, connects multiple systems together so that multiple servers can work or appear to be one machine. Due to the limited processing power of a single machine, this technology can integrate limited resources into a huge resource to meet the demand for greater computing or storage performance, thereby improving system stability and the data processing and service capabilities of the network center. In view of this advantage, cluster technology is popular, and commonly used tools such as Hadoop, spark, and flink are typical applications of cluster technology. Existing mass data analysis is based on clusters composed of a large number of nodes, and with the continuous update and development of technologies...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/00G06F11/30
Inventor 任睿田帅王磊詹剑锋
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com