Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Memory management method and device

A memory management and memory technology, applied in the field of big data, can solve the problems of reduced utilization, unavailability, waste of resources, etc., and achieve the effect of reducing memory waste

Pending Publication Date: 2021-11-26
QINGDAO HAIER TECH +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, if the memory configuration is too high, although the job will not report any errors, it will cause other jobs to be unable to use this part of the memory
Because this part of the memory is occupied and cannot be used, the utilization rate is greatly reduced, resulting in a waste of resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Memory management method and device
  • Memory management method and device
  • Memory management method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0021] Combine below Figure 1-Figure 3 Describe the memory management method and device of the present invention. figure 1 is a schematic flow chart of the memory management method provided by the present invention, such as figure 1 As shown, the present invention provides a memory management method, including:

[0022] 101. Obtain job information of each job, where the job information in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a memory management method and device. The method comprises the steps of obtaining job information of each job, wherein the job information comprises a currently-used memory and a currently-configured maximum memory; according to the currently used memory and the maximum memory, determining a current memory use ratio of each job; and determining a target job to be subjected to memory configuration adjustment according to whether the current memory use ratio of each job exceeds a first threshold and / or the duration of the current memory use ratio of each job exceeding the first threshold. According to the method, whether memory configurations of all jobs have problems or not can be monitored through the memory use proportion, and for problematic jobs, a user can be timely informed to modify the memory configurations or optimize job logic. And in combination with visual modes such as mails, a user can be informed of problem work in time, so that the memory waste condition of the big data cluster can be reduced.

Description

technical field [0001] The invention relates to the field of big data, in particular to a memory management method and device. Background technique [0002] The current big data computing framework mainly includes MapReduce, Spark, and Flink. Other types of computing frameworks are mostly derived products based on these three. For MapReduce jobs, you can use the two configuration parameters mapreduce.map.memory.mb and mapreduce.reduce.memory.mb to control the memory usage of map / reduce. For Spark-type jobs, you can use --driver-memory, --executor-memory to set the memory size of the driver / executor. For Flink-type jobs, you can set the memory usage size of jobmanager / taskmanager through –yjm, -tjm. [0003] These three jobs provide configurations, allowing users to flexibly set the memory size required by the job, but for inexperienced personnel, it is easy to set unreasonable settings. If the memory resource setting is insufficient, the job will report an error and fail...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50G06F11/30
CPCG06F9/5016G06F11/3037
Inventor 董涛
Owner QINGDAO HAIER TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com