Interactive large data analysis query processing method

Abstract

The invention provides an interactive large data analysis query processing method. The interactive large data analysis query processing method comprises steps that, 1), a query request generated by a user or a machine is submitted to an SQL query analyzer; 2), the query quest is analyzed by the SQL query analyzer into a logic plan; 3), the logic plan is read by an optimizer and is converted into a physical plan, and interaction between a storage engine and a specific data source is realized; and 4), the physical plan is performed by a performing engine, and meta-data information is provided based on a meta-database query performing framework. For real-time interactive large data set analysis blank, P-byte data is processed under the condition of low delay response through the method according to design, multiple functions of other SQL-on-Hadoop engines are integrated, the query speed is greatly improved, and thereby Hadoop data query and enterprise-level large data analysis can be rapidly and efficiently carried out for enterprise users.

Description

technical field [0001] The invention relates to the technical field of big data information retrieval, in particular to an interactive big data analysis query processing method. Background technique [0002] Big data refers to data sets that cannot be acquired, managed and processed within a certain period of time using traditional and commonly used software technologies or tools; now the focus of the term "big data" not only represents the definition of data scale, but also represents It marks the entry of information technology into a new era, represents the technical challenges and difficulties brought by explosive data information to traditional computing technology and information technology, represents the new technologies and methods required for big data processing, and also represents the big data analysis and New inventions, new services and new development opportunities brought about by applications. [0003] With the rapid development and popularization of compu...

AI Technical Summary

Problems solved by technology

[0004] In the case of fixed conditions such as server hardware and database configuration, the query efficiency of traditional query methods will continue to decrease as the amount of data increases, the response time of user queries will become slower and slower, and even the database may become unusable due to queries The problem of big data is that the data query process is slow and the query method is non-interactive. Therefore, we need a method that can quickly analyze large data sets and interactively query data

Images