114 results about "Bitmap index" patented technology

Systems and methods for accessing data stored in a data array, mapping the data using a bitmap index, and processing data queries by determining positions of query attributes in the bitmap index and locating values corresponding to the positions in the data array are described herein.

A system and method for enhancing bitmap indexing representation of a dataset, which comprises a plurality of cases and features, each case characterized by one or more values of each feature. Currently, the bins vector for each case in the dataset, is a binary array, which is a bitmap indexing representation of each respective feature of the case. The system and method enhance the bitmap indexing by padding each bins vector. The padding is carried out by identifying all target bit locations with a ‘1’ value and replacing at least one ‘0’ bit adjacent to a target bit location with a non-zero numerical value, thereby creating a padded bitmap index. The padding factor may be based on any mathematical or statistical factor concerning population or subpopulation relevant to each of the features of the dataset.

A method, apparatus, and article of manufacture for optimizing a star join operation in relational database management systems (RDBMS). A cross-product is generated from a plurality of dimension tables referenced by the star join. The join columns of the cross-product are then hashed to create a hash-row value. Using the hash-row value, a Star Map is probed to determine whether a record exists in a fact table that corresponds to the cross-product, wherein a first portion of the hash-row value is used to select a row of the Star Map and a second portion of the hash-row value is used to select a column of the selected row of the Star Map. The fact table is accessed to perform a merge join with the cross-product when the selected column of the selected row of the Star Map indicates that the record exists in the fact table.

A method, system and article of manufacture for query execution management in a data processing system and, more particularly, for managing execution of queries having one or more related query conditions. One embodiment provides a method for managing execution of a query against data of a database. The method comprises receiving a current query against the data of the database, the current query including a plurality of query conditions, for each query condition of the plurality of query conditions, determining whether a previously generated dynamic bitmap index can be re-used for the query condition of the current query, the dynamic bitmap index having been previously generated for a previous query condition associated with a previous query executed against the data of the database; and if the dynamic bitmap index has been generated for the previous query condition, retrieving the dynamic bitmap index, and determining a query result for the current query using all retrieved dynamic bitmap indexes.

Dynamic processor allocation is implemented based upon bitmap data density. A bitmap index is used to process the query. A bitmap is created for the query. The bitmap is partitioned into single I / O operations. A variable partition size is provided based upon data density. Data density for each partition is calculated. Processors are assigned based upon data density of each partition. Then the partitions are processed and query results are returned.

This invention introduces a new concept called virtual construct intervals (VCI), where each predicate interval is decomposed into one or more of these construct intervals. These VCIs strictly cover the predicate interval. Namely, every attribute value covered by the predicate interval is also covered by at least one of the decomposed VCIs, and vice versa. Each construct interval has a unique ID or interval coordinate and a set of endpoints. A construct interval is considered activated when a predicate interval using it in its decomposition is added to the system. The predicate ID is then inserted into the ID lists associated with the decomposed VCIs. To facilitate fast search, a bitmap vector is used to indicate the activation of VCIs that cover an event value. The challenge is to find an appropriate set of construct intervals to make predicate decomposition simple and, more importantly, to build efficient bitmap indexes. Because each construct interval covers only a small range of attribute values, the invention also uses bitmap clipping to cut unnecessary bitmap storage. To facilitate bitmap clipping, the invention introduce the covering segment concept. Bit positions outside a covering segment are pruned.

Systems and methods of generating and using a bitmap index are disclosed. The bitmap index stores bit strings. In a particular implementation, each bit string in the bitmap index corresponds to an attribute of an audience. Each location in each bit string corresponds to the same audience member. The bitmap index supports parallelized and distributed execution of queries with respect to data indexed by the bitmap index. For example, the described bitmap index can be used to index advertising data, healthcare data, financial data, etc.

The invention belongs to the technical field of a database, and discloses a gray-code-based distributed data layout method. The method comprises the steps of: dividing a range of each attribute into a plurality of equal portions; encoding according to a gray code order; marking a certain attribute value of a tuple comprising a plurality of attributes through the gray codes of the equal portions of the attribute value, namely an index code of the attribute value; forming an index key value of the tuple by mixing the index code of each attribute value in the tuple, wherein the tuple achieves the distributed data layout according to the order of the gray codes, the distributed data layout is deployed on a distributed system, the bitmap index of content perception is achieved at a host computer terminal of the system and the content perception is stored in a file name, and physical storage of data and statistical index of the data are achieved on a slave terminal of the system. The invention also discloses a query method employing the database formed by means of the method. The data layout obtained by the method can meet the requirements of data processing such as exact matching search, range search, multi-dimensional search, multi-attribute search and aggregated analysis, and the method is high in disc access efficiency.

The embodiment of the invention provides a bitmap index compressing method and device. Bitmap information of segments of bitmap indexes corresponding to an index key word is obtained according to preset segment width segments, deviation values of effective values of the bitmap information of the segments are stored according to the corresponding binary storage digits, compression information of the bitmap information of the segments is obtained, and due to the fact that only the deviation values of the effective values in the bitmap information of the segments are stored, storage space of the bitmap indexes can be reduced. Correspondingly, the embodiment of the invention provides a bitmap index decompressing method and device, storage bits of binary representation information of the deviation values of the effective values are utilized for obtaining the deviation values of the effective values in the compression information of the bitmap information of the segments, the bits corresponding to the deviation values of the effective values in initialized bitmap information are arranged to be the effective values, decompression information corresponding to the compression information of the bitmap information of the segments is obtained, the decompression complexity of the bitmap indexes can be reduced, and the searching query efficiency of the bitmap indexes is improved.

The invention relates to an easily compressed database index storage system using fragments and a sparse bitmap, and corresponding construction, scheduling and query processing methods. The index storage system specifically comprises a fragment type data storage structure and an index storage structure of a sparse Hash bitmap for fragment data, wherein the sparse Hash bitmap is summary knowledge for data required to be indexed and can serve as a method for extracting other summaries for the characteristics of the fragment data. By adoption of a filtering index system and the construction and query processing methods, the high-efficiency equivalent query of mass data can be realized, the disk read-write frequency of the data is greatly reduced, and occupation of an index space is extremely low. Compared with a B+ tree index, the invention is more applicable to query of historical mass data. Based on the system and the methods, a database and indexes can be conveniently compressed and stored, and a fragment compression packet can be filtered, so that the decoding overhead of compression of the database is greatly reduced.

This invention introduces a new concept called virtual construct intervals (VCI), where each predicate interval is decomposed into one or more of these construct intervals. These VCIs strictly cover the predicate interval. Namely, every attribute value covered by the predicate interval is also covered by at least one of the decomposed VCIs, and vice versa. Each construct interval has a unique ID or interval coordinate and a set of endpoints. A construct interval is considered activated when a predicate interval using it in its decomposition is added to the system. The predicate ID is then inserted into the ID lists associated with the decomposed VCIs. To facilitate fast search, a bitmap vector is used to indicate the activation of VCIs that cover an event value. The challenge is to find an appropriate set of construct intervals to make predicate decomposition simple and, more importantly, to build efficient bitmap indexes. Because each construct interval covers only a small range of attribute values, the invention also uses bitmap clipping to cut unnecessary bitmap storage. To facilitate bitmap clipping, the invention introduce the covering segment concept. Bit positions outside a covering segment are pruned.

A method includes receiving, at a computing device comprising a processor, a request to determine bicliques in a graph, where the graph includes a first set of nodes, a second set of nodes, and a set of edges, each edge in the set of edges connecting a node in the first set of nodes to a node in the second set of nodes. The method also includes determining at least one biclique based on querying a bitmap index representing the graph, where the bitmap index includes a plurality of bit strings corresponding to the first set of nodes, and where a value stored in a particular location in each bit string indicates whether an edge connects a first node corresponding to the bit string to a second node corresponding to the particular location.

The invention discloses a graph data storing method and a subgraph enquiring method based on external memories. The invention belongs to the technical field of database. The graph data storing method comprises steps that: (1) graph data formats are unified into a standard graph data format; (2) the edges of the graph data are classified and stored according to the tag information of the starting point and the end point of each edge of the graph data, and a B+-Tree index is established for each class of edges; (3) the points of the graphs are divided into a plurality of domains according to the tag information of each point of the graph data, wherein the points in a same domain orderly corresponds to a bit according to the sequence of identifiers; then, according to the tag information of the starting points and the end points of the edges, a bitmap index is established for each class of edges in the step (2); and (4) a starting point data histogram and an end point data histogram are established for each class of edges. The subgraph enquiring method comprises steps that: first, a subgraph to be enquired is decomposed; then the sub-modules obtained through decomposition are enquired; and the enquiry results are integrated. The methods provided by the invention have characteristics of high enquiring efficiency and good extensibility.

The embodiment of the invention discloses an information processing method and device. The method comprises the steps of obtaining a user information list; establishing a bitmap index comprising a plurality of bitmap vectors according to the user information list; establishing a user identity array according to the user information list; receiving a query request sent by a client; querying the user information list to obtain a query result, according to the bitmap index, the user identity array and at least one attribute value; and sending the query result to the client. In the mode, a secondary index for ad-hoc query can be realized; and the method and the device can be commonly applied to various databases, the databases do not need to be modified, and original data insertion and deletion processes are not changed, so the follow-up maintenance is facilitated.

A method includes receiving a query directed to a bitmap index. The bitmap index includes a plurality of bit strings, where a value stored in a particular location in each of the bit strings indicates whether a corresponding signal associated with a signal source has been received, and where the bitmap index includes at least a first metadata value associated with each of the bit strings, at least a second metadata value associated with each of a plurality of signal sources, or both. The method also includes generating a query execution plan for the query and generating query results based on the query execution plan. The query results identify at least one signal associated with at least one bit string of the plurality of bit strings, at least one signal source of the plurality of signal sources, and at least one of the first metadata value or the second metadata value.

An apparatus and method for Hybrid Automatic Repeat reQuest (HARQ) signaling in a broadband wireless communication system are provided. A communication method of a Base Station (BS) includes transmitting a Fixed Allocation (FA) message containing bitmap index information for all Mobile Stations (MSs) allocated with fixed resources; and transmitting bitmap information indicating whether packets to be transmitted to the MSs are Retransmission (Rtx) packets or new packets. Accordingly, when HARQ is carried out using a fixed resource allocation scheme, a bitmap is used to indicate whether HARQ packets transmitted in a current frame are new packets or Rtx packets. Therefore, a receiving end can be prevented from incorrectly decoding the packets.

The invention discloses a time-bitmap-based audio / video data access method, which comprises the following steps: dividing disk space into an index area and a data area in advance, wherein the index area is used for storing time-bitmap indexes and primary indexes, and each data bit in the time-bitmap index corresponds to one primary index respectively; and dividing the data area into a plurality of blocks, wherein each block corresponds to one secondary index. The audio / video data access procedures comprise writing of videotape sections, searching of videotape sections and reading of videotape sections. The method, disclosed by the invention, is applicable to large-capacity storage equipment, can eliminate space debris generated during the storage process, prevents the stored data from being lost in abnormal power failure, at the same time, reads data according to blocks, and is favorable for prolonging service life of the disk.

The application discloses a data storage method and data storage devices and a storage medium, and belongs to the technical field of information processing. The method includes when at least one datarecord is acquired, determining bitmaps of label values in each bitmap index partition, which is included by an bitmap index, through a preset mapping / reducing model on the basis of the at least one data record to realize storage of the at least one data record into a corresponding bitmap index partition. The bitmap index includes at least one bitmap, each bitmap corresponds to one of the label values, thus a carrying body identifier with label values can be found through the bitmap index on the basis of the label values, and efficiency of carrying out data querying on the basis of the label values is improved. In addition, the bitmaps of the label values in each bitmap index partition can be determined in parallel through the preset mapping / reducing model, and efficiency of storing data is improved.

The invention discloses an indexing method of a distributed column storage system. The method comprises the steps of obtaining distribution characteristics of each column of data, and setting a domain value and a division rule of each column of data; obtaining a continuous data area after division according to the domain value and the division rule; establishing an area coding vector corresponding to each data area; performing statistic calculation to obtain statistic information of each column of data, and combining the statistic information with the corresponding area coding vector to obtain an area coding vector with the statistic information; and performing data indexing by taking the area coding vector with the statistic information as a bit vector of a bitmap index. According to the indexing method of the distributed column storage system, a division mode of the grouping bitmap index conforms to a query filtering condition by setting the domain value and the division rule of each column of data; and by calculating the statistic information of the column data, the accuracy of grouping bitmap query is improved, the statistic information of the data in the column storage system can be reserved, and the integrity of data information is ensured.

The invention discloses a full-text database architecture, a bitmap index creating method, a data query method, a server and a medium. The full-text database architecture of the invention comprises: a data module, which is used for recording database table names of a full-text database and index field information thereof; a storage module, which includes a plurality of independent data blocks, and used for storing data and index information of database tables; a loading module, which is used for loading the data of all the database tables into the data blocks according to the index information in the metadata module, and when one of the data blocks is written fully, marking the data block as a steady state; an index creation module, which is used for generating an index of the data block marked as the steady state; and a query module, which is used for carrying out searching in the metadata module according to a query statement, searching indexes of all the data blocks if a matching index field is found, and then determining, according to the index of the data block, whether the data block meets query conditions. The architecture greatly increases the query speed and database throughput.

Systems and methods for accessing data stored in a data array, mapping the data using a bitmap index, and processing data queries by determining positions of query attributes in the bitmap index and locating values corresponding to the positions in the data array are described herein.

The invention discloses a gene variation data distributed storage method and architecture. The method comprises a distributed data storage process, a distributed bitmap index creation process and a distributed query retrieval process; and the architecture comprises a distributed column storage module, a distributed bitmap index module and a query retrieval module. A new column storage engine kuduis adopted for performing data distributed storage, and distributed local bitmap indexes are established for samples, so that the problem of low random data access performance of an existing HDFS scheme is effectively solved; the problem of poor batch analysis performance of an HBase scheme is solved; a storage architecture model is simplified; the limitation problem that a genotype query tool depends on multiple tools is solved; and through a distributed local bitmap index scheme, high concurrency is realized and the expandability is improved.

The invention discloses a bitmap index method and an intelligent card, which relate to the field of database technology. The bitmap index method can solve the technical problems of a great amount of calculation data of the existing bitmap index method in the intelligent card and more occupied memory resources by the bitmap index operation. The bitmap index method provided by the invention comprises the following steps that: an effective inquiry region of a target bit string is limited; and the bitmap index operation is carried out in the effective inquiry region of the target bit string. An operation system of the intelligent card in the embodiment of the invention is provided with a bitmap index unit, the bitmap index unit comprises: a limiting module, which is used for limiting the effective inquiry region of the target bit string; and an index module, which is used for carrying out the bitmap index operation of the effective inquiry region of the target bit string. The bitmap index method and intelligent card are mainly applied in the bitmap index.

The embodiment of the invention discloses a data storage method and device, and relates to the technical field of computers. The method comprises the steps that various pieces of data to be stored toa database are acquired, wherein each piece of data comprises a numeric value corresponding to a preset column name of the data base; at least one data column is determined from the various pieces ofdata, at least one data column file is generated, and one data column in the at least one data column is stored in each data column file; bitmap index files of the various data column files are generated; and the various data column files, the various bitmap index files and a target corresponding relation are stored, wherein the target corresponding relation is the corresponding relation among thedata column files, the bitmap index files of the data column files and preset column names corresponding to the data columns stored in the data column files. By applying the scheme to store the data,high concurrency access on the data column files can be well supported.

Under block-aware encoding, a bitmap represented by atoms comprises a series of bitmaps for each data block in a database. Each bitmap in the series is referred to herein as a block bitmap. Each block bitmap may have a different number of bytes or bits. Gaps are represented in atoms using a pair of numbers referred to as a gap code. A gap code includes a block-skip code and slot-skip code. A block-skip code represents how many block bitmaps to advance to reach a subsequent block bitmap; a slot-skip code represents how many bytes to advance within the block bitmap to reach a byte with at least one bit set. A gap code is represented by bit positions within a byte, with some bit positions allocated to represent the block-skip code and some to represent the slot-skip code. The allocation is adjusted dynamically during encoding and decoding.

Methods and a device for providing a compressed index of binary records. A method includes: sorting the records by content of a predetermined field of the record, indexing the field from one of the records in a line of a bitmap index, compressing bits in a column of the bitmap index by replacing a group of successive bits with a code, where the sorting includes the steps of assigning, for each record, a hash bucket of a hash table on a basis of a locality sensitive hash function on the contents of the predetermined field, so that the probability for two of the records to be assigned to the same has bucket increases with the similarity of the contents of the predetermined field between the records, and where at least one step of the computer implemented method is executed on a computer device.

The invention discloses a block chain light client verification and query method based on a bitmap index. The method comprises a bitmap index and transaction verification information generation step,a query request step, a block scanning step, a query processing step, a data verification step and a query result calculation step. In the bitmap index and transaction verification information generation step, index structures and verification information are constructed for different transaction types. In the query request sending step, the light client sends a query request to the full node; Inthe block scanning step, all the blocks needing to be inquired are quickly scanned by all the nodes according to the bitmap index structure; In the data query step, the whole node obtains a query result set through query and returns the query result set to the light client; In the data verification step, the light client verifies the result set returned by the full node to obtain a correct query result set; And the step of calculating the query result is that the light client performs calculation based on the correct query result set to obtain a final query result.

Dynamic processor allocation is implemented based upon bitmap data density. A bitmap index is used to process the query. A bitmap is created for the query. The bitmap is partitioned into single I / O operations. A variable partition size is provided based upon data density. Data density for each partition is calculated. Processors are assigned based upon data density of each partition. Then the partitions are processed and query results are returned.

The embodiment of the invention provides a disk storage method and device, a message pushing method and device and an electronic device. The disk storage method include acquiring key word of a messageto be stored; conducting repetition judgement operation on the message stored and to be stored based on the key word according to the bitmap index of the stored message using the Bloom filter, wherein the stored message is saved in the disk with its bitmap index stored in the random access memory; processing the message to be stored according to the repetition judgement result. The embodiment ofthe disk storage method and device, message pushing method and device and electronic equipment judges the message repetition according to the bitmap index using the Bloom filter, and performs repetition removal or information storage operation according to the repetition judgement result, which can complete searching among voluminous message under low random access memory occupation condition, andrealize highly efficient voluminous message repetition-free storage.