Real-time analysis and processing method and system for streaming data

A technology of real-time analysis and processing methods, applied in the field of data analysis, can solve the problems of difficult development, visual display of data data, high development cost of data processing system, etc., and achieve the effect of improving development efficiency and improving efficiency

Inactive Publication Date: 2017-09-08
北京思特奇信息技术股份有限公司
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is to provide a real-time analysis and processing method and system for stream data in view of the problems of high development cost, high development difficulty, low business configuration efficiency and inability to visualize data in a timely manner for data processing systems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time analysis and processing method and system for streaming data
  • Real-time analysis and processing method and system for streaming data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] The principles and features of the present invention are described below in conjunction with the accompanying drawings, and the examples given are only used to explain the present invention, and are not intended to limit the scope of the present invention.

[0052] Such as figure 1 As shown, it is a schematic flowchart of a method for real-time analysis and processing of streaming data provided by an embodiment of the present invention. The method includes:

[0053] S101. Obtain the graphical processing rules configured by the graphical configuration tool, and convert the processing rules into SQL statements;

[0054] S102. Obtain stream data that needs to be analyzed and processed in real time;

[0055] S103, extract key fields from stream data;

[0056] S104, drawing a wide table according to key fields;

[0057] S105. Perform real-time analysis and processing on the flow data according to the sql statement and the wide table.

[0058] The method for real-time ana...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a real-time analysis and processing method and system for streaming data. The method comprises the steps of obtaining a graphical processing rule configured by a graphical configuration tool, and converting the processing rule into an sql sentence; obtaining streaming data which needs real-time analysis and processing; extracting keywords from the streaming data; drawing a wide table according to the key words; and performing real-time analysis and processing on the streaming data according to the sql sentence and the wide table. By virtue of the real-time analysis and processing method and system for streaming data provided by the invention, the development efficiency can be improved effectively; the processing rule is configured by the graphical configuration tool, so that whether the caliber is accurate or not and how to convert into a professional technical language is not needed to be determined by a technical staff, so that business configuration efficiency is improved; therefore, the ever-present problem that the business personnel cannot configure the business caliber independently is solved; and in addition, real-time and visual data display can be realized by the graphical configuration tool.

Description

technical field [0001] The invention relates to the field of data analysis, in particular to a method and system for real-time analysis and processing of streaming data. Background technique [0002] With the rapid development of the mobile Internet and the advent of the era of big data, more and more data needs to be analyzed and processed in real time. The existing data processing methods include the following steps: data access, data aggregation, data precipitation and data processing. Visual display, the existing data processing platform is usually a data processing platform built by combining streaming data processing methods with real-time analysis scripts according to business needs. This kind of data processing platform requires a lot of labor costs to interpret business needs and develop analysis scripts. The disadvantages of high cost and difficult development. Existing data processing platforms have the following problems when faced with large-scale data processi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/252
Inventor 陈方龙
Owner 北京思特奇信息技术股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap