Big data access method under cloud platform

Abstract

The invention provides a big data access method under a cloud platform. The method comprises the following steps of obtaining a storage load balancing strategy and data access frequency in a data access process of a cloud storage platform and distributing storage nodes according to the strategy and the frequency to optimize a server cluster. According to the big data access method under the cloud platform, which is provided by the invention, a server cluster deployment way is simplified, a user is prevented from operating the server cluster directly, and the reasonability of the storage nodes and the data stability are guaranteed.

Description

technical field [0001] The invention relates to cloud storage, in particular to a method for accessing big data on a cloud platform. Background technique [0002] Cloud storage uses technologies such as cloud computing, distributed file systems, and server clusters to aggregate various storage resources in the network to jointly provide data storage and business access functions. The current cloud storage is divided into public service storage, which provides storage services for enterprises or individuals; the other is private architecture cloud storage, which is based on storage server clusters and distributed file systems built within the enterprise, deployed in enterprise data centers or A secure node hosting place provides corresponding storage services for the enterprise itself. The cloud storage platform has high confidentiality, and the storage process does not require too many I / O operations, so it is the best choice to build a private cloud storage system to save ...

AI Technical Summary

Problems solved by technology

However, as an open source distributed storage computing framework, Hadoop also has its own shortcomings

That is, the system architecture design is complex and operation and maintenance are difficult

The use of cloud storage platforms not only requires the accumulation of knowledge in many aspects, but also requires a lot of skills to learn and master in its operation and maintenance, which limits the industry promotion and use of cloud storage platforms to a certain extent

In building a cloud storage platform, there are two problems that affect deployment and system performance: the first is that nodes are prone to failure during operation

Once a node failure occurs and cannot be prepared to deal with it in a timely manner, it will affect the storage server cluster of multiple nodes. During the construction process, each node has many repeated operations, making the construction process very cumbersome and error-prone; the second is because The nodes are all ordinary personal computers, not dedicated servers such as minicomputers or mainframes, so data is seriously affected by CPU, memory, and disk I / O during use

Images