A data processing method and device

A data processing device and data processing technology, applied in the field of data processing, to achieve the effect of enriching data cleaning and processing functions, improving data processing efficiency, and accelerating efficiency

Active Publication Date: 2019-11-08
HAIER YOUJIA INTELLIGENT TECH (BEIJING) CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems existing in big data processing in the prior art, the present invention provides a data processing method and device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method and device
  • A data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

[0047] figure 1 It is the flow chart of data processing method in the embodiment; Reference figure 1 , data processing methods include:

[0048] Step 101, setting o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device. The method comprises the steps that operation types of all links used for describing big data processing are set, and components corresponding to the operation types are set; operation types selected by a user, an execution sequence of the operation types, the components corresponding to the operation types, a component execution sequence and parameters of the components are received; a configuration file is built; the components corresponding to the operation types selected by the user are selected according to the configuration file, the components are assembled according to the parameters of the components, the assembled components form execution modules of the corresponding operation types according to the component execution sequence, an execution sequence of the execution modules is set according to the execution sequence of the operation types, and a big data processing program body is generated and executed. According to the data processing method and device, preset components can be combined according to processing requirements of different users to provide program bodies needed by processing of the different users, the steps for writing the program of related components can be omitted, and data analysis and processing efficiency can be greatly improved.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data processing method and device. Background technique [0002] At present, when analyzing big data, the data storage and analysis platform built with the help of the open source project Hadoop is used to store device and user data. It receives and stores 860 million pieces of data every day. It has been running smoothly for more than three years and has accumulated a large amount of data. When the device reports the data description file, describe the data items and meanings reported by the device. Different devices use multiple sets of different codes to describe the device data items. The data stored on the big data platform comes from nearly a hundred different devices (different ID files). When processing data, the big data platform uses the distributed data processing interface of the Hadoop system to write data analysis and processing tasks. The written tasks need to be...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/445
CPCG06F9/44505
Inventor 郭昊
Owner HAIER YOUJIA INTELLIGENT TECH (BEIJING) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap