Data preprocessing system and method

Abstract

The invention provides a data preprocessing system and method, and relates to the technical field of smart home, and the system comprises a data obtaining module which obtains the operation state dataof all smart home equipment, and indoor environment data and outdoor environment data collected by a plurality of sensing equipment arranged indoors; a data cleaning module used for carrying out datacleaning on the operation state data according to a first preset rule and generating an equipment state data set, carrying out data cleaning on the indoor environment data according to a second preset rule and generating an indoor environment data set, and carrying out data cleaning on the outdoor environment data according to a third preset rule and generating an outdoor environment data set; and a data storage module used for adding the equipment state data set, the indoor environment data set and the outdoor environment data set into a user behavior data set and storing the user behavior data set for further data mining. The data format and content are adjusted, so that the data better meets the mining requirement, and the accuracy and effectiveness of the mined data are guaranteed.

Description

technical field [0001] The invention relates to the technical field of smart home, in particular to a data preprocessing system and method. Background technique [0002] In the era of big data, the emergence of massive data enables people to obtain unprecedented large-scale samples when dealing with computing problems, but at the same time they have to face more complex data objects. In the real world, the original data is obtained from various actual application systems. Since the data of each actual application system lacks a unified standard definition, and the data structure is also quite different, there is a large inconsistency in the data between the systems. , often cannot be used directly. At the same time, due to defects in the design of the actual application system and some human factors in the use process, some data attribute values ​​may be lost or uncertain in the data records, and necessary data may be missing, resulting in incomplete data. Data mining cann...

AI Technical Summary

Problems solved by technology

[0002] In the era of big data, the emergence of massive data enables people to obtain unprecedented large-scale samples when dealing with computing problems, but at the same time they have to face more complex data objects

In the real world, the original data is obtained from various actual application systems. Since the data of each actual application system lacks a unified standard definition, and the data structure is also quite different, there is a large inconsistency in the data between the systems. , often cannot be used directly

At the same time, due to defects in the design of the actual application system and some human factors in the use process, some data attribute values ​​may be lost or uncertain in the data records, and necessary data may be missing, resulting in incomplete data. Data mining cannot be performed directly, or the mining results are unsatisfactory

Images