352 results about "Query plan" patented technology

Method and apparatus for a searchable data service are described. The searchable data service may be implemented as a Web service with a Web service interface that allows the storage of locators and other attributes associated with entities stored in a data store in a searchable index. The attributes may be expressed as {name, value} pairs. The interface may allow client applications to query the searchable index to retrieve locators for entities in the backend data store according to the attributes associated with each locator. Search speed may be automatically optimized using, for example, indexes, query planning, and parallelism. The searchable data service provides a searchable index and is not a data store per se. The searchable data service separates searching and indexing of data from the actual storage of the data.

A system and method for distributed query processing may compile and optimize query plans for incoming query requests independent of hardware configurations and / or physical locations of data partitions in a distributed storage system (e.g., a data grid). The query plan may be divided into segments, and each segment may be instantiated on a remote query processing node of the distributed system by a query coordinator node according to metadata accessed at runtime by remote sub-query operators in the query plan. The metadata may include an indication of the physical locations of data partitions in the system and may be stored on one or more of the query processing nodes. The remote query processing nodes may execute the query plan segments and return results to the requestor. Cached query plans may be re-executed without recompilation, according to current metadata, even in the event of a node failure or data partition move.

A local database cache enabling persistent, adaptive caching of either full or partial content of a remote database is provided. Content of tables comprising a local cache database is defined on per-table basis. A table is either: defined declaratively and populated in advance of query execution, or is determined dynamically and asynchronously populated on-demand during query execution. Based on a user input query originally issued against a remote DBMS and referential cache constraints between tables in a local database cache, a Janus query plan, comprising local, remote, and probe query portions is determined. A probe query portion of a Janus query plan is executed to determine whether up-to-date results can be delivered by the execution of a local query portion against a local database cache, or whether it is necessary to retrieve results from a remote database by executing a remote query portion of Janus query plan.

An information retrieval system uses phrases to index, retrieve, organize and describe documents. Phrases are extracted from the document collection. Documents are the indexed according to their included phrases, using phrase posting lists. The phrase posting lists are stored in an cluster of index servers. The phrase posting lists can be tiered into groups, and sharded into partitions. Phrases in a query are identified based on possible phrasifications. A query schedule based on the phrases is created from the phrases, and then optimized to reduce query processing and communication costs. The execution of the query schedule is managed to further reduce or eliminate query processing operations at various ones of the index servers.

Methods and systems for searching e-mails are disclosed. In one embodiment, a method for searching e-mails includes receiving input indicative of one or more search terms. A query plan is determined based on the one or more search terms. The method includes performing a search in response to the query plan to determine information related to the one or more e-mails. Then, a set of results are generated based on the information related to the one or more e-mails.

Apparatus, systems, and methods may operate to receive an original query plan, to transform the original query plan into an equivalent executable compact query plan, and to store the compact query plan on a machine readable device. Further activities may include computing maximal source sub-queries associated with the compact query plan, and computing semi-join reductions of the maximal source sub-queries to provide an executable derivative query plan, which may also be stored on a machine readable device. Additional apparatus, systems, and methods are disclosed.

A method for evaluating a user query on a relational database having records stored therein, a workload made up of a set of queries that have been executed on the database, and a query optimizer that generates a query execution plan for the user query. Each query plan includes a plurality of intermediate query plan components that verify a subset of records from the database meeting query criteria. The method accesses the query plan and a set of stored intermediate statistics for records verified by query components, such as histograms that summarize the cardinality of the records that verify the query component. The method forms a transformed query plan based on the selected intermediate statistics (possibly by rewriting the query plan) and estimates the cardinality of the transformed query plan to arrive at a more accurate cardinality estimate for the query. If additional intermediate statistics are necessary, a pool of intermediate statistics may be generated based on the queries in the workload by evaluating the benefit of a given statistic over the workload and adding intermediate statistics to the pool that provide relatively great benefit.

System and methodology for automatic tuning of database query optimizer is described. In one embodiment, in a database system having an optimizer for selecting a query plan for executing a database query, a method of the present invention is described for automatically tuning query performance to prevent query performance regression that may occur during upgrade of the database system from a prior version to a new version, the method comprises steps of: in response to receiving a given database query for execution, specifying a query plan generated by the prior version's optimizer as a baseline best plan for executing the given database query; generating at least one new query plan using the new version's optimizer; learning performance for each new query plan generated by recording corresponding query execution metrics; if a given new query plan is observed to have better performance than the best plan previously specified, specifying that given new query plan to be the best plan for executing the given database query; if a given new query plan is observed to have worse performance than the best plan previously specified, specifying that given new query plan to be a bad plan to be avoided in the future; and automatically tuning future execution of the given database query by using the query plan that the system learned was the best plan.

Improved techniques are disclosed for processing data stream queries wherein a data stream is obtained, a set of aggregate queries to be executed on the data stream is obtained, and a query plan for executing the set of aggregate queries on the data stream is generated. In a first method, the generated query plan includes generating at least one intermediate aggregate query, wherein the intermediate aggregate query combines a subset of aggregate queries from the set of aggregate queries so as to pre-aggregate data from the data stream prior to execution of the subset of aggregate queries such that the generated query plan is optimized for computational expense based on a given cost model. In a second method, the generated query plan includes identifying similar filters in two or more aggregate queries of the set of aggregate queries and combining the similar filters into a single filter such that the single filter is usable to pre-filter data input to the two or more aggregate queries.

Disclosed are embodiments of a system for optimizing query processing in a federated information system. The system may be used to identify alternative query plans in a simulated environment and to calculate cost estimates associated with the alternative query plans, based not only on database statistics and query statements, but also based on workload and processing latencies associated with specific data source and with the federated information system as a whole. In addition the calculated cost estimates may also factor in data source availability and reliability. The system may use the alternative query plans and the associated cost estimates to influence query processing in a federated information system by feeding to the federated information system query plans that allow for cost-efficient query plan-level load distribution, cost-efficient query fragment plan-level load distribution, and load distribution based upon quality of service cost constraints.

A method may include receiving a continuous query identified by an archived view. The method may also include creating the archived view, the archived view identified by a join query related to two or more archived relations associated with an application and / or generating a query plan for the continuous query. The method may also include identifying a view root operator in the archived view, the view root operator identified as a join operator in the archived view and / or identifying a query operator in the query plan that topologically precedes the view root operator. Further, the method may include constructing an archiver query for the identified query operator that topologically precedes the view root operator, executing the archiver query to obtain a result set of data records related to the application, and / or generating a snapshot output of one or more data values related to the application.

An information retrieval system uses phrases to index, retrieve, organize and describe documents. Phrases are extracted from the document collection. Documents are the indexed according to their included phrases, using phrase posting lists. The phrase posting lists are stored in an cluster of index servers. The phrase posting lists can be tiered into groups, and sharded into partitions. Phrases in a query are identified based on possible phrasifications. A query schedule based on the phrases is created from the phrases, and then optimized to reduce query processing and communication costs. The execution of the query schedule is managed to further reduce or eliminate query processing operations at various ones of the index servers.

A system and methodology for processing of path based queries is described. In one embodiment, a method for processing a path based query requesting data from a markup language document comprises steps of: parsing the path based query requesting data to generate a query tree corresponding to the path based query; identifying all paths which may be utilized for obtaining data requested by the path based query from the markup language document; creating a transformed query tree to represent all identified paths; generating a query plan for obtaining data requested by the path based query based on the transformed query tree; and executing the query plan to obtain and return the data requested by the path based query.

A self-healing database engine and optimizer framework support automatically responding to execution errors to allow continued execution of a query plan. Upon encountering an execution error, the database engine automatically initiates a rebuilding of the query plan and executes the rebuilt execution plan. If an error is encountered in the rebuilt query plan then the query implementation methods are analyzed. If a query function is identified for which an alternative implementation method is available, then this alternative implementation method is substituted to create a new query plan. The new query plan is then executed to determine if the error is corrected.

A method for performing asynchronous statistics updates in a database management system includes receiving a first query against the database, determining if present statistics related to the first query are stale and entering on a queue a request to acquire updated statistics if the present statistics are stale. The queue jobs are executed asynchronously with respect to the query request. As a result, a first query plan may be developed using the present statistics related to the first query. Thus, no delay in processing the query due to statistics updates is incurred. The first query plan may be executed and results given to the requester. At some later time, the request to acquire updated statistics related to the first query is processed asynchronously from the query request. If subsequent queries are received, the queue can delete duplicate requests to update the same statistics. Those subsequent queries can benefit from the updated statistics.

A method may include receiving a continuous query identified by an archived relation associated with an application. The method may also include generating a physical query plan. Additionally, the method may include constructing a structured query language (SQL) equivalent expression corresponding to an attribute utilized by the operator and / or constructing an output SQL expression for the identified operator based at least in part on the SQL equivalent expression. Additionally, the method may include constructing the archiver query for the identified operator based at least in part on the output SQL expression. Further, the method may include executing the archiver query to obtain a result set of data records related to the application and / or generating a snapshot output of one or more data values related to the application based at least in part on the result set of data records.

A system and method for creation, modification and / or use of plan guide(s) is provided. Plan guide(s) can be employed to optimize the performance of query(ies) when it is not possible and / or desirable to change the text of the query directly. Plan guides can be useful, for example, when a small subset of queries in a database application deployed from a third-party vendor is not performing as expected. Plan guides influence optimization of queries by attaching query hints to them. In accordance with an aspect of the subject invention, a query plan execution system is provided. The system includes an execution component, a query optimizer, and, a plan cache. Plan guides and other associated metadata are stored in a plan guide metadata store.

Techniques for managing memory usage in a processing system are provided. This may be achieved by receiving a data stream including multiple tuples and determining a query plan that was generated for a continuous query applied to the multiple tuples in the data stream. The query plan may include one or more operators. Before scheduling an operator in the query plan, it is determined when an eviction is to be performed based a level of free memory of the processing system. An eviction candidate is determined and a page associated with the eviction candidate is evicted from the memory to a persistent storage.

The present invention concerns a complex event processing (CEP) engine (1) for processing CEP queries (10) over data streams, wherein the CEP engine (1) comprises:a. a parser (100), adapted for parsing a received CEP query (10) into a logical query graph (20); andb. a translator (300), adapted for translating the logical query graph (20) into a physical query plan (30) in accordance with one of a plurality of data stream representations; whereinc. the logical query graph (20) is independent of the plurality of data stream representations.

Performing database queries. A method includes receiving a particular database query. The method further includes accessing a query plan based on the particular database query. The query plan has operators and specific operational parameters associated with each of the operators. The association of operators and specific operational parameters is specific to the particular database query. From the query plan, the method further includes instantiating a plurality of compiled code templates. Each code template includes executable code that when executed performs functionality of one of the operators from the query plan with the specific operational parameters applied in the compilation. The method further includes binding the code templates together using programmatic control flow to create a functioning program.

A method and apparatus, and article of manufacture are provided to process an abstract query of a database abstraction constructed over an underlying physical data storage mechanism. The database may comprise a single data source, or a federated source spanning multiple systems. Embodiments of the invention process an abstract query by generating an intermediate representation of the abstract query. In one embodiment, the intermediate representation comprises an abstract query plan. An abstract query plan includes a combination of elements from the data abstraction model and elements relating to the underlying physical data storage mechanism. Once generated, a back-end component may easily traverse the abstract query plan to generate an SQL statement (or other resolved query) corresponding to the abstract query. Further, once constructed, the abstract query plan provides a platform for many different optimizations that may be selected by a user or by the runtime component inspecting the abstract query plan prior to creating the resolved query.

A system and method for avoiding accessing a remote database twice, once during query execution and again to retrieve objects identified by the query, when an application requires objects on which to operate. During query plan generation in response to a request for database objects from an application, a query optimizer inserts cache operators into the candidate plans, and then a cost-benefit analysis is undertaken to identify the best plan. The best plan is then used to execute the query, with the cache operators causing objects identified during query execution to be cached locally to the requesting application as the query is being executed, thereby avoiding requiring the application to access the database after query execution.

Methods, systems, and apparatuses for improving performance of parallel database query plans are described. An exchange operator is positioned in a query tree. A child operator of the exchange operator is parallelized into a plurality of parallel child operators, each of the parallel child operators coupled to the exchange operator in a respective branch of a plurality of parallel branches of the query tree. An output of each of the plurality of parallel child operators may be buffered at the exchange operator. Furthermore, child operators of the plurality of parallel child operators may also be parallelized. Query plans of any form and containing any number of operators may be parallelized in this manner. Any number of parallel branches may be used, independent of the number of operators in the original plan. The parallelized query plans achieve effective load balancing across all branches.

A storage system for providing a preferable data access performance by performing controls considering database management system (DBMS) execution information or database process priorities, by acquiring static configurational information of a DBMS by means of a DBMS information acquisition and communication program, a DBMS information communication section, and a host information setting program; acquiring DBMS execution information by means of a query plan acquisition program, the DBMS information communication section, and a process performance management program; acquiring information on priorities of database processes given by the process performance management program; and storing them in disk I / O management information with process priorities, DBMS execution information, and DBMS data information, in which cache control section in a storage system control program controls a data cache by referring to the above information.

Disclosed are embodiments of a system for optimizing query processing in a federated information system. The system may be used to identify alternative query plans in a simulated environment and to calculate cost estimates associated with the alternative query plans, based not only on database statistics and query statements, but also based on workload and processing latencies associated with specific data source and with the federated information system as a whole. In addition the calculated cost estimates may also factor in data source availability and reliability. The system may use the alternative query plans and the associated cost estimates to influence query processing in a federated information system by feeding to the federated information system query plans that allow for cost-efficient query plan-level load distribution, cost-efficient query fragment plan-level load distribution, and load distribution based upon quality of service cost constraints.

System, methods and articles of manufacture for optimizing a query plan reuse in a database server system accessible by a plurality of client connections. An embodiment comprises providing at least one global cache storage and a private cache storage to a plurality of client connections, and coordinating utilization of the at least one global cache storage and the private cache storage to share light weight stored procedures and query plans for prepared SQL statements across the plurality of client connections via the at least one global cache storage while avoiding a spinlock access for executing the prepared SQL statements.

In some examples, a method may include receiving a continuous query, the continuous query being identified based at least in part on an archived relation associated with an application. The method may also include generating a physical query plan for the continuous query. In some examples, the method may also include identifying a query operator in the physical query plan and / or constructing an archiver query for the identified query operator. In some aspects, the method may also include executing the archiver query to obtain a result set of data records related to the application and / or initializing a state of the query operator based at least in part on the result set of data records. Further, the method may also include generating a snapshot output of one or more data values related to the application based at least in part on the state of the identified query operator.