Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Task processing method, server, client and system

A task processing and server technology, applied in transmission systems, electrical digital data processing, instruments, etc., can solve the problem of low submission efficiency

Pending Publication Date: 2021-03-23
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF9 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present application provides a task processing method, server, client and system, which are used to solve the technical problem of low submission efficiency caused by the existing way of submitting Spark tasks

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task processing method, server, client and system
  • Task processing method, server, client and system
  • Task processing method, server, client and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] figure 1 It is a schematic diagram of the basic architecture of Spark. Such as figure 1 As shown, the basic architecture of Spark may include the following components: Spark kernel, Spark SQL, Spark Streaming (Spark Streaming), machine learning library (machine learning library, MLlib), and GraphX.

[0054] Spark kernel (core): Contains the basic functions of Spark, such as task scheduling, memory management, fault tolerance mechanism, etc. RDDs are defined inside the Spark kernel, which can provide application programming interface (application programming interface, API) to create and operate RDDs, and provide underlying services for other components. Other Spark libraries are built on top of RDD and Spark kernel.

[0055] Spark SQL: is a library for processing structured data in Spark, providing SQL through Apache Hive (Hive is a data warehouse tool based on Hadoop, which can map structured data files into a database table and provide SQL-like query functions) Va...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a task processing method, a server, a client and a system, and the method comprises the steps that the server receives a first task request command submitted to a drive of an application by the client through a remote call port, and the first task request command requests to process a target task of the application, wherein the first task request command carries an access address of a service logic code of the target task; the server acquires a pre-stored service logic code of the target task according to the access address; the server processes the target task through the driver according to the service logic code of the target task; and the server returns a processing result of the target task to the client through the remote calling port. According to the embodiment of the invention, the efficiency of submitting the Spark task can be improved.

Description

technical field [0001] The embodiments of the present application relate to communication technologies, and in particular to a task processing method, server, client and system. Background technique [0002] Spark is an open source big data computing engine based on memory iteration. It supports a variety of data processing tasks, such as batch processing and stream processing, through data sharing and abstract elastic distributed datasets (RDDs). , structured query language (structured query language, SQL) query, machine learning and graph computing, etc. Because Spark has the characteristics of fast speed, high throughput, distributed expansion, etc., Spark has become the most popular big data computing engine and is widely used in big data systems in various industries. [0003] Spark is generally deployed on server clusters that use YARN to manage resources. There are two modes for submitting Spark tasks, YARN-Cluster and YARN-Client. In the YARN-Cluster mode, users ca...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/54G06F9/50G06F9/48H04L29/06H04L29/08
CPCG06F9/548G06F9/5066G06F9/4881H04L67/10G06F2209/544H04L67/01
Inventor 巴俊
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com