Multi-source data processing method and device for big data architecture and block chain

A technology of multi-source data and processing methods, applied in the field of communication, can solve the problems of high data maintenance cost and poor security.

Pending Publication Date: 2020-12-18
BOYA ZHENGLIAN (BEIJING) TECH CO LTD +2
View PDF0 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The increase in the amount of data also brings another problem: the problem of data security
Traditional databases run on a single-node server or a cluster of several servers, and the cost of data maintenance is high, and the security is not very good

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-source data processing method and device for big data architecture and block chain
  • Multi-source data processing method and device for big data architecture and block chain
  • Multi-source data processing method and device for big data architecture and block chain

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0037] Although the present invention provides method operation steps or device structures as shown in the following embodiments or accompanying drawings, more or less operation steps or module units may be included in the method or device based on conventional or creative work . In steps or structures that do not logically have a necessary causal relationship, the execution order of these steps or the modular structure of the device is not limited to the execution order or module structure shown in the embodiments of the present invention or the drawings. When the method or module structure is applied in an actual device or terminal product, it can be executed sequentially or in parallel according to the method or module structur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-source data processing method, device and system for a big data architecture and a block chain. The method comprises the following steps: performing data acquisition ona plurality of data sources and converting the acquired data into a data stream with a unified format; realizing classified caching of the data streams and providing a data stream output interface; acquiring a data stream through a data stream output interface and calling a big data open source algorithm to consume the acquired data stream; and obtaining a data stream through the data stream output interface and storing the obtained data stream to the block chain. The invention provides a unified and lightweight data processing platform capable of meeting various actual business scenes, data acquisition of different data sources is realized, and the acquired data is converted into a data stream with a unified format, so that various data query and analysis tools can be conveniently and quickly read. In addition, the classified and stored data streams can be quickly and conveniently transferred to the block chain, so that the application of the block chain is met.

Description

technical field [0001] The present invention relates to the field of communication technology, in particular to a multi-source data processing method and device for big data architecture and block chain. Background technique [0002] In recent years, with the rapid development of science and technology and the advancement of informatization construction, as small as the user cache in the background of each application program on the mobile terminal, as large as the log data stored on the server cluster to record user access and its own operating status, the data in It is generated and accumulated at the PB level all the time. The increase in the amount of data has brought about an increase in the value of the data. A large amount of data plays a vital role in the fields of user behavior analysis and system security alarms. Numerous data that are valued begin to reflect new value. [0003] But on the other hand, due to the irregular data format and random data storage in th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/25G06F16/2455G06F16/27
CPCG06F16/258G06F16/24552G06F16/24568G06F16/27
Inventor 孙圣力赖凯庭李青山司华友
Owner BOYA ZHENGLIAN (BEIJING) TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap