42 results about "Full table scan" patented technology

A system and method for providing security to a data warehousing application. A system is described having: a dimension analysis system that identifies dimensions defined for viewing data in a data warehouse; a security table generation system for automatically generating security tables for identified dimensions; a security tables scanning system for analyzing inputs within the security tables; and an end-user view definitions update system for updating the data warehouse based on inputs in the security tables.

The invention discloses an OLAP (On-Line Analytical Processing) star connection query optimizing method under a CPU (Central Processing Unit) and GPU (Graphics Processing Unit) mixing framework. The method comprises the following steps: first filtering and optimizing an OLAP star connection operation through bitmap connection indexing, buffering high-frequently accessed connecting bitmap in a GPU buffer memory, next, loading fact table external key attribute group satisfying the connecting bitmap filtering condition into the GPU buffer memory to carry out star connection filtering, and finally switching the whole table scanning of the big fact table of the internal memory into accessing randomly according to the position through filtering bitmap generated by the GPU, thereby improving the querying and treating performances of the OLAP star connection. The method improves the storage efficiency of the GPU buffer memory and the parallel processing efficiency of the GPU, and improves the OLAP querying and treating performance of the hybrid processor platform on the whole.

The invention provides a database partition method used for partitioning a database. The method comprises the following steps of: acquiring multiple required report forms; determining database partition rules according to time nodes of the multiple required report forms; and partitioning the database according to the determined database partition rules. The database partition method is used for analyzing the time nodes of the required reports and customizing the partition rules according to the time nodes, so that the data of various reports such as full network daily reports, busy hour reports and the like best accord with the characteristics of full partition table scanning strategies, the required data can be acquired during full partition table scanning, the scanned records which do not meet the search conditions reach the minimum, the time required for searching and scanning is reduced, the full table scanning efficiency is improved, and many original conditions of needing index scanning are changed into conditions of full table scanning. The invention also provides a database partition device.

The invention provides a method and a system by using temporary performance objects for enhanced query performance, and discloses a method and an apparatus, including computer program products, implementing and using techniques for optimizing query performance in a relational database management system. A query is received at the relational database management system. It is determined whether the query is expected to be a long-running query. In response to determining that the query is expected to be a long-running query, a full table scan is started in order to fetch records needed to satisfy the query from the relational database management system. In parallel with conducting the full table scan, a performance object is built, which is capable of satisfying the query in the relational database management system. In response to completing the construction of the performance object prior to completing the full table scan, the full table scan is stopped and the newly built performance object is used instead to satisfy the query.

A system for predicting an Elapsed Response Time (ERT) of a query to be executed in a production database comprises a db emulator module for creating an emulated database based on empty schema derived by extrapolating statistical information from the development environment, an execution plan generation module for generating a query execution plan of the query, a query classifying module for classifying each sub-query based on the operational parameters into at least one of a full table scan, a primary index scan, a fast index scan and a non-unique index scan, a time determination module for executing each sub-query on the emulated database to determine an Input Output (IO) access time and an operator time, and an ERT prediction module for predicting the ERT of the query to be executed in the production database based on the I0 access time, the operator time, and a network time.

The embodiment of the invention discloses a data table query method and device, terminal equipment and a storage medium. The method comprises the steps that a query operation, aiming at a target datatable, in a current affair is received, the query start time of receiving the query operation is recorded, whether or not a modification operation aiming at the target data table exists in the influence time period from the affair start time of the current affair to the query start time is judged, the total number of records affecting the target data table is determined, and the total number of target records of the target data table is determined according to the total number of initial records of the target data table predetermined at the query start time and the total number of the affectedrecords. Accordingly, the total number of the target records of the target data table is determined through the total number of the initial records of the predetermined target data table and the total number of the affected records, the problem that full table scanning is conducted when data table query is conducted, and the query efficiency is low is solved, the data table query efficiency is improved, occupied memory resources are reduced, and the overall performance of a database is optimized.

The invention relates to a database substring filtering index system and a method for constructing and inquiring the database substring filtering index system. The database substring filtering index system is characterized by comprising a fragmented data storage module and a filtering index storage module, wherein the fragmented data storage module is used for storing input data in a fragmented manner and extracting the characteristics of the substrings describing the fragmented data from each fragmented data to construct one or more substring characteristic filter bitmaps; and the filtering index storage module is used for storing substring characteristic filter bitmaps to be substring characteristic indexes; the invention further provides the method for constructing and inquiring the substring filtering index system. The database substring filtering index system has a rational design, solves the problem that the traditional index space is large, and the compressed data is difficult to index, achieves the function of efficiently inquiring substrings and keywords of massive text data, thereby greatly reducing the disk reading-writing times to data and improving full table scanning performance of the database.

A system and method for providing security to a data warehousing application. A system is described having: a dimension analysis system that identifies dimensions defined for viewing data in a data warehouse; a security table generation system for automatically generating security tables for identified dimensions; a security tables scanning system for analyzing inputs within the security tables; and an end-user view definitions update system for updating the data warehouse based on inputs in the security tables.

The big data storage, analysis and query system comprises a distributed data storage framework, a distributed data analysis framework, a distributed data search framework and a data visual display framework; the distributed data storage architecture is used for providing high throughput to access data of an application program with a super-large data set; the distributed data analysis architectureis a general engine and is used for completing processing operation of large-scale data; the distributed data search architecture is used for a distributed mass data search engine in various scenes;and the data visualization display framework adopts an MVC structure. Through distributed storage and query, a plurality of nodes perform storage and querying at the same time, so that the efficiencyis improved, and the time is saved. the query in the mass data, compared with a traditional database in which full-table scanning is adopted and the time is slow, in distributed query technologies, the same data can reach the second-level speed, the analysis types are rich, various data samples are supported, distributed analysis is supported, and the analysis speed is greatly increased.

The invention discloses a netted multilevel index matching method for publishing and subscribing, and belongs to the field of information technology processing. According to the method, an improved netted multilevel index structure is proposed for the problem that full-table scanning is needed when subscribed items of a user are deleted in an existing multilevel index matching algorithm, the subscribed items can be locked with link table header pointers of the subscribed items and then deleted under the condition that the indexing efficiency is ensured, full-table scanning of multilevel indexes in the deleting process is omitted, and thus the index deleting efficiency is improved; multiple subscribed items corresponding to subscribed item attributes are defined with a doubly-linked table, identical subscribed items in the indexes are connected, and thus the index maintaining efficiency is improved; indexes are established in levels for operate operators and attribute subscribed items respectively, querying conditions can be rapidly pruned and filtered, and thus the index querying and matching efficiency is improved.

The invention discloses an index system and an index method for retrieving time sequences of ultra-large-scale data. The index system comprises a vertical hierarchical index module and a corresponding time axis index module. The vertical hierarchical index module comprises a plurality of hierarchies of indexes, each hierarchy of indexes comprises a hash function and a plurality of data sets, and data in the data sets are mapped into a corresponding next hierarchy of data set by the aid of the hash function of the corresponding next hierarchy of indexes; the time axis index module is used for creating event lists and time lists. The index system and the index method have the advantages that original big data sets can be ultimately distributed into a plurality of small data sets by means of hierarchy -by- hierarchy hash mapping by the aid of the hierarchical index module, operation such as query processing, data loading and storage optimization can be independently executed on each small data set, accordingly, the vertical hierarchical index module can be combined with the time axis index module, risks of full-table scanning operation in time sequence retrieval operation procedures can be prevented, and the time sequence retrieval speeds can be greatly increased; the index system and the index method for retrieving the time sequences of the ultra-large-scale data can be widely applied to the field of data processing.

The embodiment of the invention discloses a data table query method and device, terminal equipment and a storage medium. The method comprises the steps that a query operation, aiming at a target datatable, in a current affair is received, the query start time of receiving the query operation is recorded, whether or not a modification operation aiming at the target data table exists in the influence time period from the affair start time of the current affair to the query start time is judged, the total number of records affecting the target data table is determined, and the total number of target records of the target data table is determined according to the total number of initial records of the target data table predetermined at the query start time and the total number of the affectedrecords. Accordingly, the total number of the target records of the target data table is determined through the total number of the initial records of the predetermined target data table and the total number of the affected records, the problem that full table scanning is conducted when data table query is conducted, and the query efficiency is low is solved, the data table query efficiency is improved, occupied memory resources are reduced, and the overall performance of a database is optimized.

The invention discloses a prediction method and system of database publishing time. The method comprises the steps that a mapping list is built, and a DDL operation type received by a database is judged; whether or not the DDL operation type can give rise to whole list scanning is judged, if yes, the sum of the list capacity weight value and the list heat weight value of the database is calculated, and if not, the list heat weight value of the database is calculated; the result is used as the comprehensive weight value of the database; the publishing time of the database corresponding to the comprehensive weight value is queried in the mapping list. By means of the prediction method and system of the database publishing time, the database publishing time can be reasonably predicted, the influence of database publishing on a generating environment is reduced, the success rate of publishing is improved, and long-time blockage of the database caused by database publishing on an inappropriate occasion or other faults of the database are avoided.

Methods for optimizing query performance in a relational database management system. A query is received at the relational database management system. It is determined whether the query is expected to be a long-running query. In response to determining that the query is expected to be a long-running query, a full table scan is started in order to fetch records needed to satisfy the query from the relational database management system. In parallel with conducting the full table scan, a performance object is built, which is capable of satisfying the query in the relational database management system. In response to completing the construction of the performance object prior to completing the full table scan, the full table scan is stopped and the newly built performance object is used instead to satisfy the query.

The invention provides a method and device for determining a data comparison result, and equipment The method comprises the following steps that: data in a main table and an auxiliary table are loadedinto a data comparison table, wherein the table structure of the data comparison table comprises a main table field and a comparison field; the initial value of the comparison field corresponding toa row with the same data corresponding to the main table field in the data comparison table isupdated into a first numerical value, so that a first data comparison table is obtained; based on a main key field included in the main table field, rows of which comparison fields in the first data comparison table are not a first numerical value are grouped; the initial value of each row of comparison fields in each group is updated according to the number of rows included in each group, so that a second data comparison table is obtained; and a comparison result corresponding to each row of data inthe second data comparison table is determined according to a preset relationship between a comparison field value and the comparison result. By utilizing the method, device and equipment of the embodiment of the invention, full-table scanning can be avoided, and the comparison efficiency is improved.