Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-cluster cooperative data processing method and system, memory medium and computer device

A data processing system and data processing technology, applied in the field of data processing, can solve problems such as low data exchange efficiency and data redundancy, and achieve the effect of improving data exchange efficiency, reducing the risk of inconsistency, and good scalability

Active Publication Date: 2018-02-09
INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF4 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The embodiment of the present invention provides a multi-cluster collaborative data processing method to overcome the limitations of the cluster constructed by a single technology to deal with business scenario services, as well as the problems of data redundancy and low data exchange efficiency among various heterogeneous systems
[0008] The embodiment of the present invention also provides a multi-cluster cooperative data processing system to overcome the limitations of the cluster constructed by a single technology to deal with business scenario services, and the problems of data redundancy and low data exchange efficiency among various heterogeneous systems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-cluster cooperative data processing method and system, memory medium and computer device
  • Multi-cluster cooperative data processing method and system, memory medium and computer device
  • Multi-cluster cooperative data processing method and system, memory medium and computer device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments and descriptions of the present invention are used to explain the present invention, but not to limit the present invention.

[0035] In order to overcome the limitation of the cluster constructed by a single technology to deal with business scenario services, and the problems of data redundancy and low data exchange efficiency among various heterogeneous systems, the present invention proposes a multi-cluster collaborative data processing method. figure 1 is a schematic flowchart of a multi-cluster cooperative data processing method according to an embodiment of the present invention. Such as figure 1 As shown, the multi-cluster collaborative data processing method in the embodiment of the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a multi-cluster cooperative data processing method and system, a memory medium and a computer device. The method comprises the steps of receiving collection data and convertingthe collection data into structured data; establishing a basic data sharing area on a Hadoop cluster and loading the structured data to the basic data sharing area; and cooperatively processing the data in the basic data sharing area through utilization of the Hadoop cluster, a Hadoop online processing cluster and a plurality of distributed database clusters according to a data application scene,thereby providing corresponding data service and finishing a data processing process. According to the method, the system, the memory medium and the computer device, the data redundancy can be reducedthrough establishment of the basic data sharing area.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a multi-cluster cooperative data processing method, system, storage medium and equipment. Background technique [0002] In recent years, with the advent of the era of big data, more and more companies have realized the value of data, and hope to guide their business decisions through data analysis and data mining. Driven by these demands, big data processing technologies applicable to various scenarios have emerged. Among them, the technical ecosystem represented by Hadoop technology and the distributed Database technology is the most representative. The share nothing architecture is adopted, each node uses independent local resources, and data is generally distributed according to the distribution key Hash, and data exchange between nodes is realized through the network to achieve linear expansion of the system and efficiently process PB-level data. [0003] Al...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/24G06F16/258G06F16/27
Inventor 杨晨旭王备蔡良良阳维迅
Owner INDUSTRIAL AND COMMERCIAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com