Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Real-time industrial process big data compression and storage system and method

An industrial process and compressed storage technology, which is applied in the field of big data analysis, can solve the problems of compressed storage that is difficult to meet the demand, does not involve big data compressed storage, etc., and achieves the effects of fast online query, maximum economic benefits, and huge cost savings

Active Publication Date: 2017-01-25
NORTHEASTERN UNIV
View PDF6 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] At present, the patents on big data compression storage systems mainly include CN1853198A, which involves compressing video images and audio data, using transform coding data such as wavelet transform or DTC, and grouping transform coefficients by most significant bits and / or bit values, And the transmission of them group by group does not involve the compressed storage of real-time big data in industrial processes. As industrial enterprises pay more and more attention to the value of their data, enterprises are gradually building industrial big data collection, storage, and analysis systems. Existing It is difficult for patents to meet the demand for compressed storage of real-time big data in industrial processes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time industrial process big data compression and storage system and method
  • Real-time industrial process big data compression and storage system and method
  • Real-time industrial process big data compression and storage system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0070] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0071] In this embodiment, the mineral processing industry is monitored in real time, and a total of 14 production data are collected, including operating status data, working environment data, and material information data. The selected big data framework is Hadoop cluster, version 2.6.0, including 1 master node Master, 19 slave nodes Slave1, Slave2, ..., Slave19, the selected nodes are Ubuntu 13.04 operating system, 8G memory. Build Zookeeper+Kafka clusters in 20 servers to provide the necessary environment for real-time industrial process big data analysis and processing. Zookeeper is used for resource management, Kafka is used for data transmission, and a Topic topic i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a real-time industrial process big data compression and storage system and method and relates to the technical field of big data analysis. The system comprises a data acquisition module, a log file storage module, a memory cache module, a data compression module, a non-relational database and a rendering device. The data acquisition module is used for acquiring real-time industrial process data and synchronously sending the real-time industrial process data to the memory cache module and the log file storage module, data in the memory cache module is compressed, and the compressed data is stored in the non-relational database. By means of the real-time industrial process big data compression and storage system and method, the purpose of storing real-time industrial process big data can be effectively achieved, the real-time industrial process big data is compressed to the maximum and stored with the non-relational database, the real-time industrial process big data is effectively stored, the huge cost generated for storing the real-time industrial process big data can be reduced for enterprises, and online inquiry, calculation and the like of data can be quickly responded to.

Description

[0001] Technical field: [0002] The invention relates to the technical field of big data analysis, in particular to a real-time industrial process big data compression storage system and method. [0003] Background technique: [0004] With the continuous development of computer technology, big data has emerged as the times require, which has set off a frenzy of digital reform on a global scale. Data has become a valuable resource, and the effective storage and utilization of this resource can enable enterprises to transform and upgrade to intelligent production and intelligent decision-making. At present, real-time big data storage of industrial processes generally adopts real-time databases and relational databases. For industrial enterprises, the data scale is growing at a rapid rate. To ensure that the characteristics of the data itself are not lost and that the stored data can meet the needs of big data analysis Under the premise of this, how to maximize the storage of da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/1744
Inventor 徐泉刘文庆张鹏冉振莉王良勇许美容崔东亮
Owner NORTHEASTERN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com