Internet information statistical method and Internet information statistical system
A technology of Internet information and statistical methods, applied in the field of Internet information statistical methods and its systems, can solve the problems of not being able to provide options and interfaces, rough ranking information, etc.
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0041] figure 1 It is the flow chart of the Internet information statistics method described in this embodiment, as figure 1 As shown, the Internet information statistics method described in this embodiment includes:
[0042] S101. Divide network access data into multiple service subject data sets.
[0043] In this step, the network access data is divided into multiple business theme data sets through MapReduce according to the business theme. The network access data includes network-wide traffic data IMOS log data for data analysis, and these massive data are stored in the large-scale distributed storage system ODS. The high-speed partition processing of massive data is exactly what the MapReduce data processing mechanism is good at. This data processing mechanism can divide a large amount of data into different data sets through distributed parallel computing in a very short period of time. Therefore, this The invention adopts the MapReduce mechanism to implement the divi...
Embodiment 2
[0054] According to the same idea of the present invention, the present invention also provides an Internet information statistics system, figure 2 It is a block diagram of the Internet information statistics structure described in this embodiment, such as figure 2 As shown, the system includes: a data splitting unit 201 , a data summarizing unit 202 , and a data query unit 203 .
[0055] Wherein, the data splitting unit 201 divides the network access data into multiple business theme data sets through MapReduce according to the business theme. The network access data includes network-wide traffic data IMOS log data for data analysis, and these massive data are stored in the large-scale distributed storage system ODS. The high-speed partition processing of massive data is exactly what the MapReduce data processing mechanism is good at. This data processing mechanism can divide a large amount of data into different data sets through distributed parallel computing in a very...
Embodiment 3
[0063] The present invention also provides the Internet information statistics system realized based on the distributed data processing framework Handoop, such as image 3 As shown, the system mainly includes an upper-level business system 301 , a service layer 302 , a data mart (DM) 303 , a data warehouse (DW) 304 , and a distributed storage system (ODS) 305 . Among them, the data mart DM is implemented based on HBASE, the data warehouse DW is implemented based on HIVE, and the distributed storage system ODS is implemented based on HDFS.
[0064] Next, introduce its data processing process. First, import the network access data IMOS from the outside to the storage system ODS, and then extract the data from the ODS to the data warehouse DW through ETL. The full name of ETL is Extraction-Transformation-Loading, that is, data extraction, transformation and loading. Tools that can implement ETL include: OWB (Oracle Warehouse Builder), ODI (Oracle Data Integrator), Informatic Po...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com