Data storage method and device, apparatus and storage medium

Abstract

The invention relates to a data storage method and device, an apparatus and a storage medium, and the method comprises the steps of creating a data cluster which comprises a plurality of first attribute nodes, a plurality of second attribute nodes and a plurality of indexes; obtaining the current time and the creation time of each index in the data cluster, and for each index, determining whetherthe index meets a first migration condition or not according to the current time and the creation time of the index; creating a migration index list, and adding the indexes meeting the first migrationcondition in the data cluster into the migration index list; and determining the index meeting a second migration condition in the migration index list as a target index, and migrating the index dataof the target index from the first attribute node to the second attribute node. According to the present invention, the index data meeting the preset condition in the data cluster can be migrated, sothat the index data with different attributes can be separately stored, and the storage cost is reduced.

Description

technical field [0001] The present invention relates to data storage technology, in particular to a data storage method, device, equipment and storage medium. Background technique [0002] Elasticsearch is a distributed storage search engine. For a distributed storage system, data is stored scattered on multiple nodes. Elasticsearch nodes can have various attributes, including data nodes, master nodes, ingest nodes, and so on. In the native Elasticsearch storage solution, by configuring the relevant properties of the node, you can set its node type, and by configuring the path parameters, you can set its corresponding data directory. One or more Elasticsearch nodes will form an Elasticsearch cluster, and the data written to the cluster will be stored on each data node in units of indexes and fragments, and the data will be automatically balanced on each data node. [0003] Since Elasticsearch supports a variety of data types, such as row storage, column storage, inverted i...

AI Technical Summary

Problems solved by technology

[0003] Since Elasticsearch supports a variety of data types, such as row storage, column storage, inverted index, spatial index, etc., the storage space consumption is relatively large; generally, the original data written to Elasticsearch requires storage space that is multiple times the size of the data itself, and the data of the Elasticsearch cluster The storage mode is single, and the data cannot be stored separately according to the heat of data access. If all the data is stored on a disk with better performance, the cost of data storage will be high

Images