Data classification optimization method based on Cockroach DB bottom layer key value

A technology of data classification and optimization methods, applied in the field of distributed databases, can solve problems such as untargeted optimization of underlying RocksDB storage, and achieve the effect of promoting application value, improving read and write performance, and optimizing read and write amplification problems.

Inactive Publication Date: 2021-02-12
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF7 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The distributed database CockroachDB stores node health data, monitoring indicator data, transaction record data, RAFT logs, RAFT state machines, Range data, copy data and other system data and table data in multiple underlying RocksDB storage with the same characteristics on multiple nodes In this case, it is impossible to optimize the underlying RocksDB storage according to the data with different read and write characteristics

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data classification optimization method based on Cockroach DB bottom layer key value
  • Data classification optimization method based on Cockroach DB bottom layer key value
  • Data classification optimization method based on Cockroach DB bottom layer key value

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0037] In the CockroachDB-based underlying key-value data classification optimization method of the present invention, an enumeration type DataType is added to the head of the RPC request in the distributed database CockroachDB, and the underlying data classification identifier and table identifier are parsed out in the request processor. And table identification and passed to the storage engine, the storage engine adopts different data optimization strategies for different types of read and write characteristics.

[0038] like figure 1 As shown, the enumeration type DataType is defined, and the data is divided into six categories: transaction record data, range data, RAFT data, time series data, table data and other data.

[0039] Among them, the data is divided into six categories: transaction record data, range data, RAFT data, time series data, table data and other data. According to their different read and write operation characteristics in the database, the configuratio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an underlying key value data classification optimization method based on Cockroach DB, and belongs to the technical field of distributed databases. According to the invention,an enumeration type DataType is newly added in the header of an RPC request in a distributed database Cockroach DB, and underlying data classification identifiers and table identifiers are analyzed ina request processor to obtain data types and table identifiers and transmitted to a storage engine; and the storage engine adopts different data optimization strategies for different types of read-write characteristics. According to the invention, the read-write amplification problem of the database can be optimized, so that the read-write performance of the database is improved, and the method has very good popularization and application values.

Description

technical field [0001] The invention relates to the technical field of distributed databases, and specifically provides a method for classifying and optimizing bottom-level key-value data based on CockroachDB. Background technique [0002] CockroachDB is a NewSQL distributed transaction database implemented based on the "Spanner: Google's Globally-Distributed Database" paper proposed by Google. It not only has the ability to store and manage massive amounts of data, but also supports the ACID and SQL standard query languages ​​of traditional databases, and also has distributed features such as high availability, high concurrency, and scalability. [0003] The overall architecture of the distributed database is divided into SQL layer, transaction layer, distribution layer, copy layer, and storage layer. The SQL layer translates and parses the SQL statements of the client into key-value pair (KV) operations, and the transaction layer guarantees multiple key-value pair operatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/957
CPCG06F16/957
Inventor 苑晓龙孙兴艳孙思清高传集周恒
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap