Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A batch processing method for large computer data

A big data and batch processing technology, applied in the fields of electrical digital data processing, special data processing applications, computer security devices, etc., can solve the problems of user loss, difficult realization, and low scheduling rate of massive data resources, and eliminate uncertainty. sexual effect

Pending Publication Date: 2019-03-26
贵州斯曼特信息技术开发有限责任公司
View PDF5 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (1) Traditional big data security protection technologies cannot protect sensitive information and sensitive data within the big data platform, which may easily cause data leakage and cause losses to users
[0005] (2) When analyzing big data, the analysis time is long, the work efficiency is low, and it is not easy to implement, and batch analysis errors are prone to occur
[0006] (3) The traditional resource scheduling module has a low scheduling rate for massive data resources, resulting in a slow batch processing rate and wasting more time
[0007] When analyzing big data, the existing technology does not use the granular computing method to analyze the inaccurate solution of big data problems, and convert the input of the problem from the most fine-grained raw data to information granular representation, under the premise of retaining the information and value contained in the data , greatly reducing the amount of data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A batch processing method for large computer data
  • A batch processing method for large computer data
  • A batch processing method for large computer data

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0180] Example 1. Optimization problem in granularity space. Using optimization theory to describe the selection problem of granular layer, determine the calculation granularity of effective solution, so as to obtain effective solution in the shortest time.

[0181] Definition 1. The effectiveness of the solution can be defined by a two-tuple SolutionEffectiveness=(GM(R), T u ).R is the result of calculation, GM(R) is the granularity measure of the result, T u is the time-bound requirement. If the GM GM(R) of a solution and the time to obtain the solution is less than T u , then this solution is valid and is called an efficient solution.

[0182] In order to select a "suitable" granular layer from the domain-oriented granular structure for calculation, in order to reduce the actual time and space consumption of calculation, it is necessary to perform granular space optimization. Granular space optimization is to find such granular Layer Layer i :

[0183] MaxGM (Layeri)

...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of big data batch processing and discloses a batch processing method for large computer data, which inputs customer data through a data input module by using a data input device. The main control module utilizes the scheduling algorithm to schedule the data resources to be processed through the resource scheduling module, and the resource scheduling module adopts the Min-Min scheduling algorithm, which utilizes batch processing program to schedule processor to batch process the job to be processed by batch processing execution module; Encryption module is used to encrypt the large data by the encryption program. The analysis module analyzes the big data by using the analysis program, and the data storage module stores the big data resources by using the memory. large data information content is displayed by a display module using that display. The invention does not need to obtain large data from a large amount of large data in a distributeddatabase, so the invention is short in time and easy to realize.

Description

technical field [0001] The invention belongs to the technical field of big data batch processing, in particular to a batch processing method of computer big data. Background technique [0002] Big data includes structured, semi-structured and unstructured data, and unstructured data is increasingly becoming the main part of data. According to the survey report of IDC, 80% of the data in enterprises are unstructured data, and these data are growing exponentially by 60% every year. Big data is just a manifestation or characteristic of the Internet at the present stage. There is no need to myth it or maintain awe of it. Against the background of technological innovation represented by cloud computing, these things seem difficult to collect The data used and used are beginning to be easily utilized. Through continuous innovation in all walks of life, big data will gradually create more value for human beings. The generation of big data analysis is aimed at IT management. Enter...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F21/62G06F21/60G06F16/22G06F16/2453G06F16/2455G06F16/25G06F16/27G06F16/2458G06F9/50
CPCG06F9/505G06F21/602G06F21/6218G06F2216/03
Inventor 张辉
Owner 贵州斯曼特信息技术开发有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com