Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Hadoop-based network data mining and analysis platform and method thereof

A network data and platform technology, applied in network data retrieval, database distribution/replication, electronic digital data processing, etc., can solve the scalability barriers of traditional relational data management systems, storage performance and scalability constraints restrict the effective application of data bottlenecks, To deal with problems such as new data power failure, to achieve the effect of reducing the amount of model calculation, easy portability, and improving algorithm efficiency

Active Publication Date: 2017-07-07
南方电网互联网服务有限公司
View PDF8 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In addition, the rapid growth of various network users and network access points and the emergence of various new network media have caused a rapid expansion of the data volume of various unstructured and semi-structured network data, while traditional relational data management systems (parallel The scalability of the database) has encountered unprecedented obstacles, making it incapable of handling new types of data
Due to the huge scale, various forms and dynamic changes of data generated in various industries, if the traditional relational database is still used to store these unstructured data, its storage performance and scalability will become the bottleneck restricting the effective application of data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hadoop-based network data mining and analysis platform and method thereof
  • Hadoop-based network data mining and analysis platform and method thereof
  • Hadoop-based network data mining and analysis platform and method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] It should be pointed out that the following detailed description is exemplary and intended to provide further explanation to the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0046] It should be noted that the terminology used here is only used to describe specific implementations, and is not intended to limit the exemplary implementations according to the present application. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and / or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and / or combinations thereof.

[0047] figure 1 It is a schematic structural diagram of a Hadoop-based network data mining and analysis platf...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a Hadoop-based network data mining and analysis platform and a method thereof. The platform comprises a data collection layer, a data storage layer, a business application layer and a user layer, wherein the data collection layer adopts a distributed directional collection system architecture, collects original network data by taking terminal sites in different networks as basic task units of network data collection, and transmits the data to the data storage layer in a gathering manner, wherein each basic task unit adopts independent collection rule and policy; the data storage layer is used for finishing gathering, storage and original processing of the original network data and providing different types of function call services; the data storage layer is realized by adopting a Hadoop framework; and the business application layer is used for calling and analyzing the network data processed by the data storage layer to realize separation of a public component and a personalized business application component, and transmitting a result after network data analysis to the user layer for performing real-time display.

Description

technical field [0001] The invention belongs to the field of network data processing, and in particular relates to a Hadoop-based network data mining and analysis platform and a method thereof. Background technique [0002] In recent years, "big data" has become a hot spot in the scientific and technological circles and business circles. Data has become an equally important strategic resource as natural resources and human resources. of great importance. If these big data are effectively organized and used, it will play a huge role in promoting the development of society and economy. These rapidly increasing data mainly come from people's daily life, especially the Internet has become my country's largest public information distribution center and social group platform. Compared with traditional communication media such as newspapers, wireless radio and television, online media has comprehensive characteristics such as low barriers to entry, super-large-scale information, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/258G06F16/27G06F16/958
Inventor 朱世伟赵燕清阎淮海鞠镁隆于俊凤魏墨济李晨李思思徐蓓蓓李宪毅王爱萍
Owner 南方电网互联网服务有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com