A parallel distributed big data architecture construction method and system
A construction method and big data technology, applied in the construction method and system field of parallel distributed big data architecture, can solve problems such as complex, unsuitable for large-scale data query, difficult to guarantee data consistency, etc., to ensure load balance and improve data The effect of query ability
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0030] refer to Figures 1 to 2 , which is the first embodiment of the present invention, which provides a method for constructing a parallel distributed big data architecture, including:
[0031] S1: The grid is established by the grid unit 100, and the data is sequentially stored in the grid according to the time stamp.
[0032] (1) Create a grid.
[0033] Define a total of n*m data, and calculate the average density ρ of each layer of data nodes;
[0034] According to the average density ρ, the data area of each layer is divided into grids, and the density ρ of the data nodes in the grid is judged i is close to the mean density ρ, define |ρ i -ρ|≥0.01 is not close, if not, the grid is divided according to the area of the data node;
[0035] If it is close, continue to divide the data area of each layer according to the average density ρ.
[0036] Among them, n is the number of data layers, m is the number of nodes in each layer of data, ρ i Represents the i-th d...
Embodiment 2
[0091] Different from the first embodiment, this embodiment provides a parallel distributed big data architecture construction system, including:
[0092] The grid unit 100 is used to establish a grid, and store the data into the grid in sequence according to the time stamp;
[0093] The computing unit 200, connected with the grid unit 100, is used for computing grid data correlation and data node sampling time interval, wherein the grid data correlation includes grid spatial correlation C N , data acquisition time correlation D T Correlation with data collection location D L .
[0094] The transmission unit 300 is connected to the grid unit 100 and the computing unit 200 respectively, and is used for allocating grid data to the data storage unit 400;
[0095] The data storage unit 400 is used for storing grid data; the data storage unit 400 includes a RAC database and a Teradata database.
[0096] It should be appreciated that embodiments of the present invention may be i...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com