Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and apparatus for testing data processing system

A data processing system and technology of a processing system, which is applied in the field of test data processing systems and can solve problems such as task processing delays

Active Publication Date: 2019-02-15
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF2 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In this way, since the problem can only be solved after the data processing system goes online, but solving the problem after the problem occurs will cause serious delays in task processing, so it is urgent to provide a method for testing the data processing system in advance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and apparatus for testing data processing system
  • Method and apparatus for testing data processing system
  • Method and apparatus for testing data processing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0039] In order to facilitate the understanding of the embodiments of the present invention, the system architecture involved in the embodiments of the present invention and concepts of terms involved are firstly introduced below.

[0040] The embodiment of the present invention can be applicable to a data processing system, and the data processing system can be a big data processing system, such as figure 1 As shown, as for the Spark big data processing system (also referred to as a Spark cluster), the system can be deployed on multiple computer nodes, and use multiple computer nodes to process large-scale data in parallel (which can be referred to as big data for short).

[0041] Abnormality: Abnormal phenomena, errors such as ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application provides a method and apparatus for testing a data processing system, belonging to the field of big data processing. The method comprises when the data processing system to betested is tested, determining a target application in the data processing system to be tested, then obtaining the exception rules for the target application, in accordance with that exception rule, generating at least one set of distribute exception data, then for each set of distributed exception data, according to the value range of each configuration parameter in a plurality of preset configuration parameters corresponding to the target application, controlling the target application program to process the distribution abnormal data to obtain the target values of the preset configuration parameters corresponding to the preset performance parameters when the performance values of the preset performance parameters of the data processing system to be tested meet the preset conditions, andthen generating a test report based on the distribution of the distribution abnormal data and the target values of the preset configuration parameters. With this application, the method of testing thedata processing system is provided.

Description

technical field [0001] The present application relates to the field of big data processing, in particular to a method and device for testing a data processing system. Background technique [0002] In recent years, with the rapid development of technologies such as the Internet, e-commerce, and the Internet of Things, the speed of data generation has shown an explosive growth trend. These data have the characteristics of fast generation and large scale. In order to tap the huge commercial value hidden behind these massive data, many data processing systems have been developed, such as Hadoop, Spark, Storm, Flink, etc. These data processing systems are respectively focused on in different processing scenarios. When the data processing system is processing data, errors such as memory overflow, I / O exception, and task no response often occur, and these errors will directly cause the task execution failure of the data processing system. [0003] In related technologies, general...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F11/36
CPCG06F11/3672
Inventor 刘新春时金魁许利杰
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com