Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for performing data operations on real-time data streams and data sets

A data stream and data set technology, which is applied in the fields of electronic digital data processing, structured data retrieval, special data processing applications, etc., can solve problems such as inability to meet real-time performance and long-term data delay.

Pending Publication Date: 2021-12-07
BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this case, in scenarios with high real-time requirements, the calculation scheme based on time windows will cause long data delays, which cannot meet the real-time requirements

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for performing data operations on real-time data streams and data sets
  • Method and device for performing data operations on real-time data streams and data sets
  • Method and device for performing data operations on real-time data streams and data sets

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028]Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0029] It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0030] figure 1 A schematic diagram 100 of a first embodiment of a method f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and device for performing data operation on real-time data streams and data sets. The method comprises steps of responding to a received operation request, and obtaining a to-be-analyzed data set corresponding to the operation request and the real-time data streams corresponding to the operation request; partitioning the data set to be analyzed and the real-time data stream to generate each sub-data set and each sub-data stream; loading each sub-data set and the sub-data stream corresponding to the sub-data set to each task node corresponding to the partition sub-thread; in response to the fact that loading of each task node is completed, preset operation being executed on the sub-data streams and the sub-data sets in the task node, and an operation result of the task node being generated; and determining a final operation result corresponding to the operation request according to the number of the task nodes and the operation result of each task node. According to the method, a data operation method for partitioning the real-time data stream and the large data set and then carrying out association calculation is realized, calculation accuracy is ensured, and efficiency of data real-time calculation is improved.

Description

technical field [0001] The present application relates to the field of computer technology, specifically to the field of data processing technology, and in particular to methods and devices for performing data operations on real-time data streams and data sets. Background technique [0002] In real-time computing tasks, there will be a scene where a certain link needs to filter the entire data stream with unified data rules. The traditional technical solution is to use broadcast broadcasting technology to copy and distribute a small data set to each task task node , each task node directly reads data from the local during the data calculation process, but the existing real-time computing framework has limitations on the variable data size of the broadcast broadcast. When the size of the data set exceeds a certain scale, it cannot be calculated in real time Use the broadcast technology to broadcast public variables. [0003] For the above scenarios, the existing real-time co...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F16/27
CPCG06F16/24568G06F16/278
Inventor 朱时涛申作军郭旭波
Owner BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com