Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Stream processing method and system for online filling of network missing data

A missing data, stream processing technology, applied in the field of computer networks, can solve the problems of increasing the cost of filling time, unbalanced gradient update, and low filling accuracy.

Active Publication Date: 2021-08-24
HUNAN UNIV
View PDF7 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] In view of the above defects or improvement needs of the prior art, the present invention provides a stream processing method and system for online filling of missing data in the network. The matrix is ​​used as input, and each sequence needs to train parameters from scratch, which leads to the technical problem of greatly increasing the time cost of filling, and the existing machine learning-based filling method does not train a filling model with fixed parameters through historical data. It is applicable to the technical problems of filling network data, as well as the technical problems of slow convergence speed of the model due to failure to consider the characteristics of network data conforming to the heavy-tailed distribution, and the defect of unbalanced gradient update in the process of training the model, resulting in the final The technical problem of low filling accuracy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stream processing method and system for online filling of network missing data
  • Stream processing method and system for online filling of network missing data
  • Stream processing method and system for online filling of network missing data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other. .

[0066] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising", "comprising" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device comprising a series of elements Not only those elements are included, but also other elements not expressly listed or inherent in such proc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a stream processing method for online filling of network missing data, which comprises the following steps: in the process of performing stream processing on the network missing data, extracting time and space information contained in a monitoring data matrix sequence in a previous time period through a feature extractor and a gating circulation unit; a context vector which retains effective information in historical data can be obtained, the context vector is combined with spatial feature vector information corresponding to network missing data at the current moment, and a combined vector is input into a pre-trained missing data generation model so as to obtain a current monitoring data matrix filled with the network missing data. According to the stream processing method and system for online filling of the network missing data provided by the invention, the spatial feature vector corresponding to the network missing data is combined with the context vector related to the previous historical data, so that the spatial-temporal information of the network data in the previous time period is fused, and the precision of online filling of the network missing data can be effectively improved.

Description

technical field [0001] The invention belongs to the technical field of computer networks, and more specifically relates to a stream processing method and system for online filling of network missing data. Background technique [0002] With the rapid development of communication technology, the scale of the network that needs to be maintained is also increasing. In the process of network operation and maintenance, it is necessary to measure network monitoring data for subsequent tasks such as anomaly detection, root cause analysis, and traffic prediction. However, the cost of performing network-wide measurement on a large-scale network is very high, and frequent measurement will also occupy a large amount of network resources. [0003] Studies have shown that network monitoring data has temporal correlation and spatial correlation, so the current mainstream processing method is to measure part of the data and restore the unmeasured data according to the correlation. Since t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06F16/2455G06F16/2458G06N20/00
CPCG06F16/215G06F16/24568G06F16/2474G06N20/00
Inventor 谢若天谢鲲李肯立文吉刚
Owner HUNAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products