Data snapshot method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a data snapshot method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: responding to a received data snapshot instruction, enabling a first thread to create a data snapshot thread, and updating the identification information of to-be-snapshot data; in response to detecting that the to-be-snapshotdata is modified, setting identification information of the modified to-be-snapshot data as latest identification information; and the data snapshot thread performing snapshot processing on the dataaccording to the identification information, performing data snapshot on the to-be-snapshot data with the non-latest identification information, and performing lag data snapshot on the to-be-snapshotdata with the latest identification information. According to the technical scheme, snapshot operation of total data can be achieved under the condition that main thread service is not affected, timedelay jitter caused by Fork operation in the prior art is eliminated, meanwhile, memory consumption can be greatly reduced, the memory utilization rate is increased, resource cost is reduced, and thenthe stability of Redis product service is effectively guaranteed.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of data processing, and in particular to a data snapshot method, device, electronic equipment, and computer-readable storage medium. Background technique [0002] With the development of data technology, the application of high-performance Redis database is more and more widely. Redis is an open source, written in ANSI C language, supports network, can be based on memory and can be persistent log-type Key-Value database , in order to ensure efficiency, the data in Redis is first cached in memory, and then periodically writes updated data to disk or writes modification operations to additional record files, also known as persistence or data snapshot operations. In the current Redis internal mechanism, persistence, data snapshot, and master-slave synchronization all rely on the operating system's Fork and COW (copy on write, copy-on-write) mechanisms, where both Fork and COW generate new ...

AI Technical Summary

Problems solved by technology

However, this mechanism has the following defects: 1. The Fork system call will consume a lot of time, resulting in second-level jitter in the main thread service; 2. The COW copy is based on a physical memory page and will be released when the child process exits. During the snapshot, a large amount of additional memory is copied, causing memory consumption to double

The above defects will have a great impact on the stability of Redis products and services, and also limit the memory utilization of Redis in the virtual machine environment, and also increase resource costs.

Images