General byte stream parsing system of big data collection and realization method of system

A byte stream and big data technology, applied in transmission systems, electrical components, etc., can solve problems such as waste of collectors and difficult analysis, and achieve the effect of standardization of the byte stream analysis process

Active Publication Date: 2016-12-21
袭明科技(广东)有限公司
View PDF4 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Analyzing data is a crucial link in data collection. In the era of big data, data are widely distributed and have complex and diverse structures, which brings grea

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • General byte stream parsing system of big data collection and realization method of system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Such as figure 1 As shown, the byte stream parsing component of the present invention consists of two parts: a byte stream preprocessor and a two-dimensional table parser. The byte stream preprocessor automatically checks the data according to the agreement and automatically selects the corresponding decompression and decryption methods to preprocess the data according to the compression and encryption methods; the two-dimensional table parser defines the two-dimensional table according to the byte stream format, and uses row and Rowgroups store data information, in which rowgroups are used to store collection data, and columns are used to indicate the data structure of the byte stream; the two-dimensional table parser divides the byte stream into multiple fields, and then divides the divided byte stream according to the data type Parse it into corresponding information, and read the parsed structured information through a two-dimensional table.

[0030] The byte strea...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of computer application, especially a general byte stream parsing system of big data collection and a realization method of the system. The system is composed of a byte stream pre-processor and a two-dimensional table analyzer. According to conventional automatic verification data, the byte stream preprocessor preprocesses data by selecting a corresponding decompression and decryption method automatically according to a compression and encryption manner; the two-dimensional table analyzer defines a two-dimensional table according to the byte stream format, data information is stored in rows and row groups, columns are used to mark the name, data type and length of each field, after that the byte stream of data is read in, the byte stream is segmented into multiple fields according to the bit length defined in each column, and the segmented byte stream is parsed into corresponding information according to data types; and data or set information read by the rows or multiple rows in the row group is read, and concrete field information of the data rows is read via the field name of the column. According to the invention, repeated defining, compiling and parsing are avoided, and the system and method are suitable for parsing data of complex and diversified structures in big data collection in a unified manner.

Description

technical field [0001] The invention relates to the field of computer application technology, in particular to a general big data collection byte stream analysis system and its implementation method. Background technique [0002] Analyzing data is a crucial link in data collection. In the era of big data, data are widely distributed and have complex and diverse structures, which brings great difficulties to analysis. Each collection needs to write analysis codes according to different data structures. This duplication of work is a great waste for collectors. Contents of the invention [0003] One of the technical problems solved by the present invention is to provide a general big data collection byte stream analysis system, which makes the byte stream analysis process more standardized and controllable. [0004] The second technical problem that the present invention solves is to provide a kind of implementation method of general big data acquisition byte stream analysis...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/06
CPCH04L63/0428H04L63/123H04L69/18
Inventor 吴建州季统凯
Owner 袭明科技(广东)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap