Big data cleaning method and device, computer equipment and storage medium

Abstract

The invention discloses a big data cleaning method and device, computer equipment and a storage medium, and belongs to the technical field of big data. According to the method, the mapping relation between the service data types and the cleaning rules is configured, the cleaning rule matching table is generated, the distributed cluster is constructed, the distributed cluster comprises a plurality of sub-servers, each sub-server correspondingly processes one kind of service type data, the cleaning rules are distributed to the corresponding sub-servers according to the service data types, and the cleaning rules are distributed to the corresponding sub-servers according to the service data types. a target service data type is determined corresponding to the original data, a target sub-server corresponding to the target service data type is searched and the data cleaning is performed on the original data in the target sub-server to obtain cleaned data. In addition, the invention also relates to a block chain technology, and original data can be stored in a block chain. According to the method, automatic cleaning of different types of service data is achieved by constructing the distributed cluster, the method has high universality and adaptability, and meanwhile unified management of data cleaning rules is facilitated.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to view more

AI Technical Summary

Benefits of technology

In this patented technology, there are two main ways that large amounts (large) amount of data needs to be processed: one way involves collecting it into smaller units called chunks or partitions within an organization's own system; another approach uses multiple servers connected together by communication networks like LAN/WLAN connections between them. These techniques aim to automate efficient cleanup processes on these larger datasets without requiring manual intervention from IT staff members.

Problems solved by technology

The technical problem addressed in this patented text relates to how current methods involve manually developing cleanings that are specific based on factors like size, time-to-effectiveness, and other aspects related to ensuring proper performance during use across various industries such as healthcare systems (HCPs). This can lead to wasted effort and resource allocation due to manual processes being performed separately from each industry's needs.

Images