Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data processing method and device based on molap

A technology of data processing and data cube, which is applied in the field of data processing, can solve problems such as delays of several seconds, tens of seconds or even tens of minutes, high server configuration requirements, and inability of non-technical personnel to query, etc., to improve query response Speed, the effect of simplifying the data query process

Active Publication Date: 2019-12-31
GUANGZHOU SHIYUAN ELECTRONICS CO LTD
View PDF1 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The traditional MOLAP engine is limited by hardware and software resources, and can only process gigabytes or <10 terabytes of data, and when calculating multi-dimensional cube data, it requires high server configuration
At the same time, for real-time query of massive data based on MOLAP, the structured query language (SQL on Hadoop) scheme based on distributed system infrastructure is often used. On the one hand, for some columnar databases, usually only fast queries can be performed based on row keys, while column-level queries can only be used in specific query scenarios
In addition, when querying, it needs to involve the writing of SQL statements, so that non-technical personnel cannot query

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device based on molap
  • Data processing method and device based on molap
  • Data processing method and device based on molap

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0025] figure 1 It is a flow chart of a MOLAP-based data processing method provided by Embodiment 1 of the present invention. The method provided in this embodiment can be executed by a MOLAP-based data processing device, which can be implemented by means of software and / or hardware, and integrated in a server. refer to figure 1 , the method provided in this embodiment may include:

[0026] S110. Create a data cube according to the fact table and the dimension table.

[0027] Among them, the fact table and the dimension table are stored in the data warehouse of the big data platform. The dimension table stores the values ​​of all attributes under the dimension and the ID of each record. For example, the sales region dimension includes information such as the region ID, the province to which the region belongs, the city to which it belongs, and the county to which it belongs. Taking Guangzhou Tianhe District as a value under the sales region dimension as an example, the rec...

Embodiment 2

[0041] figure 2 It is a flowchart of a MOLAP-based data processing method provided by Embodiment 2 of the present invention. The data processing method provided in this embodiment is optimized on the basis of the data processing method provided in the foregoing embodiments. refer to figure 2 , the data processing method provided in this embodiment specifically includes:

[0042] S210. Create corresponding fact tables and dimension tables according to item requirements for fact tables and dimension tables in a preset data analysis model.

[0043] Specifically, the preset data analysis model includes item requirements for fact tables and dimension tables, where the item requirements may include dimension values ​​and dimension levels of dimension tables, and fact data attributes of fact tables (such as sales amount, etc. ). In addition, the data analysis model may also include metrics during data precomputation (such as aggregated maximum values, etc.). The specific conte...

Embodiment 3

[0130] Figure 7 It is a schematic structural diagram of a MOLAP-based data processing device provided by Embodiment 3 of the present invention. refer to Figure 7 , the data processing apparatus provided in this embodiment specifically includes: a cube creation module 701 , a precalculation module 702 and a saving module 703 .

[0131] Wherein, the cube creation module 701 is used to create a data cube according to the fact table and the dimension table; the pre-calculation module 702 is used to perform data pre-calculation on all possible combinations of dimensions based on the data recorded in the data cube; the preservation module 703 uses Save the pre-calculated results to the open source database, so that the query results can be determined according to the pre-calculated results when querying.

[0132] The technical solution provided by this embodiment creates a data cube through the fact table and the associated dimension table, performs data pre-calculation on all pos...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention discloses a multidimensional online analytical processing (MOLAP)-based data processing method and apparatus. The data processing method comprises the steps of creating a data cube according to a fact table and a dimension table; performing data pre-calculation on all possible combinations of dimensions according to data recorded in the data cube; and storing a pre-calculation result in an open-source database, so as to determine an inquire result according to the pre-calculation result in inquiry. Through adoption of the method, the existing data inquiry solution can be optimized, and non-technical staff can inquire in massive data.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a data processing method and device based on MOLAP. Background technique [0002] Online Analytical Processing (OLAP) system is the most important application of data warehouse system, which is specially used to support complex analysis operations, focusing on decision support for decision makers and senior managers. OLAP can perform complex query processing of a large amount of data according to the requirements of analysts, and provide the query results to decision makers in an intuitive form, so that they can accurately grasp the operating conditions of enterprises (companies), understand the needs of objects, and formulate correct solutions. scheme. [0003] According to the data storage format of its memory, OLAP system can be divided into three types: relational OLAP (Relational OLAP, ROLAP), multidimensional OLAP (Multidimensional OL...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/28G06F16/2453
CPCG06F16/2453G06F16/283
Inventor 李寅威
Owner GUANGZHOU SHIYUAN ELECTRONICS CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com