177 results about "Time series database" patented technology

Systems and methods for time series data error detection, correction, and transformation may detect gaps and anomalies in time series data, such as from a meter device, and may correct the gaps and adjust the anomalies prior to long-term record storage. Data forecasting may be used to correct the errors in the time series data. The error corrected data may be regarded as an actual set of time series data and become a base data set against which additional heuristic projections are generated. In addition, the time series data may be transformed into any number of physical and virtual device hierarchies that represent the underlying data source configurations, and may then be stored in an analytical database for further analysis. The hierarchies may be irregular and may change over time.

If the operations manager makes configuration changes of the log database through configuration management interface, the configuration management interface records configuration information thereof on an management database. Furthermore, when log data of a monitoring target device are collected in a log collection unit through the log database, a log data volume analysis unit reads utilization information of data volume of the log database, and records the reading on a management database. A configuration changes proposal generating unit automatically generates configuration changes proposal of the log database based on the information recorded on the management database. Moreover, a metrics value for evaluating time required for configuration changes, a metrics value for evaluating performance in connection with the configuration changes, and a metrics value for evaluating the data volume in connection with the configuration changes are calculated, and an effect and an influence of changes proposal are presented to the operations manager.

The invention provides an access trend prediction based time series database cache management method. The method comprises: on the basis of providing a data access cache by a conventional database, establishing a time series database dual-cache mechanism; and in addition to a conventional cache, adding a prediction data cache for predicting pre-loading of access data to load the access data into a memory before data access so as to increase the query speed. The method is used for effectively managing the time series database data cache to improve the efficiency of data cache hitting and system data query. A system analyzes recent access data and predicts a data access trend, and the prediction data is loaded to increase the data access speed. The method is suitable for time series database data cache management and is used for improving the efficiency of time series database data query.

A method for performing queries on a distributed time series data storage system is presented. The time series data storage system has a time series database that stores data blocks containing time stamped data across a plurality of computing devices. The system also includes an index database that stores an index associated with the time stamped data in each data block. The method includes the steps of sending a query, requesting indices, returning the indices, preparing a sub-query, forwarding the sub-query to an evaluator, evaluating the sub-query, performing a logical operation on each sub-query's result, receiving the sub-results at an output handler, and combining the sub-results.

A method and apparatus for constructing a memory-based database service platform, in which database can be on-loaded and off-loaded or unloaded as needed, and can reserve schedule and size of memory and other resources, including CPUs, network, backup, mirroring and recovery recourses. With the service platform, multiple different types of databases can be chosen by specifying data storage type and data operation interfaces, such as Relational Database (RDB), Biometric Database (BDB), Time Series Database (TDB), Data Driven Database (DDDB) and File-based Database (FDB) etc. Database types can be chosen either by user directly or by platform automatically or semi-automatically based on data types and data operation characteristics.

The invention relates to a processing method and device of monitoring data. The method comprises the steps of collecting the monitoring data of a target data source; writing the monitoring data into a subscribed message distributed type publishing system; reading the monitoring data from the subscribed message distributed type publishing system through a consumer machine; making the consumer machine obtain target data source identification in the monitoring data; making the consumer machine obtain a distributed type time series database corresponding to the target data source according to the target data source identification, wherein the distributed type time series database is a stand-alone version; sending the monitoring data to the corresponding distributed type time series database through the consumer machine and storing the monitoring data. Through the processing method and device of the monitoring data, the stability and high usability of a distributed type time series database cluster are ensured.

A computer-implemented method for employing deep learning for time series representation and retrieval is presented. The method includes retrieving multivariate time series segments from a plurality of sensors, storing the multivariate time series segments in a multivariate time series database constructed by a sliding window over a raw time series of data, applying an input attention based recurrent neural network to extract real value features and corresponding hash codes, executing similarity measurements by an objective function, given a query, obtaining a relevant time series segment from the multivariate time series segments retrieved from the plurality of sensors, and generating an output including a visual representation of the relevant time series segment on a user interface.

The invention discloses a time series data aggregation retrieval method and device and a medium. The method comprises a database for storing the time series data, and comprises the steps of firstly, optimizing the time series database, adding a statistical level concept for the data storage, carrying out the hierarchical fragmentation storage of the time series data, and providing the data structure support for the optimization of the time series data aggregation retrieval; when the time series data is subjected to statistical query, applying different query algorithms according to different statistical query requests, and subjecting the time series data to the statistical query in combination with the hierarchical fragmented data storage structure. When the data is written into the database, the data only needs to be subjected to hierarchical fragmentation statistics according to the determined hierarchical fragmentation rule so as to record the data characteristics, and conditions for future query and retrieval are not considered in the writing process; during the statistical query, according to the retrieval conditions, the original data and all levels of statistical data are used, a statistical result is calculated after optimization and combination without depending on mass original data, so that the data access amount is reduced, and the retrieval efficiency is improved.

The invention discloses a data inquiry method and device based on a mixed storage architecture. The method comprises the following steps: initiating the mixed storage architecture and providing a uniform SQL (structured query language) engine for the mixed storage architecture; acquiring an SQL data inquiry request compiled according to an SQL with a preset uniform standard; parsing the acquired SQL data inquiry request according to the provided uniform SQL engine; performing corresponding data operation on different databases according to a parsing result and acquiring corresponding data operation results; combining a plurality of acquired data operation results to form a total inquiry result. The method provided by the invention supports unified SQL data inquiry in mixed storage of a relational database and a time series databases, therefore, a uniform standard access of the mixed storage architecture is realized, the data inquiry efficiency of the mixed storage architecture is effectively improved, the data inquiry cost is greatly lowered, and various business applications, such as a big data analysis and the like, are effectively supported.

The invention discloses an integrated resource monitoring system based on an open source cloud platform, containing a data acquisition module and a data display module, wherein the data acquisition module adopts monitoring tools of Zabbix, Telegraf and Influxdb, the Zabbix realizes large-scale monitoring deployment and collection of any monitoring item through various monitoring ways and functions, the Telegraf uses a plug-in mechanism to collect a plurality of pieces of resource monitoring information and store the information in the Influxdb time series database; the data display module introduces Grafana open source monitoring software, accesses monitoring information of the plurality of platforms of Zabbix, Influxdb and OpenStack by using existing plug-ins and secondary development plug-ins by the plug-in mechanism of Grafana, collects multi-dimensional monitoring items, and displays the monitoring data through Grafana's rich dashboards and charts. The invention performs multi-dimensional and comprehensive real-time monitoring on resource information such as the service state and health state of the cloud platform, the virtual machine and the application of the cloud platform.

The invention provides a time series data storing and querying method and a storing and processing platform, and relates to the technical field of databases. The time series data storing and queryingmethod and the storing and processing platform are applied to a database, and the database comprises an indexing file and a data file. The time series data storing method comprises the following steps: receiving time series data to be stored and compressing the time series data to obtain compressed time series data; writing the compressed time series data to the tail of a data file; determining the offset, timestamp and data length of the compressed time series data; and when the compressed time series data are written successfully, constructing an index entry according to the timestamp, offset and data length, and writing the index entry to the tail of an index file. An embodiment of the invention aims to provide the time series data storing and querying method and the storing and processing platform so as to rapidly store, query and process a bulk of time series data, and time series database service which is high in disaster tolerance ability, convenient to use and efficient is provided.

The invention discloses a time sequence database system based on column storage. The time sequence database system comprises: an SQL (Structured Query Language) parser, wherein the SQL parser is usedfor converting SQL into an abstract syntax tree through parsing; a logic plan generator which is used for generating a logic query plan by utilizing the abstract syntax tree, wherein the logic query plan is expressed by a relation algebra; a query optimizer which is used for optimizing the query plan by utilizing the state information of the database and outputting the rewritten query plan; an executor which is used for executing the query plan in a block iteration mode and is responsible for interacting with the storage layer to obtain data queried by a user; and a storage engine which is used for storing data in a row and column manner. According to the invention, data in the database is stored in a column mode; based on combination of column storage and rocksdb, high-speed insertion data characteristics and an indexing function of a traditional database are achieved, in the aspect of an executor, part of operators are improved to be deeply combined with characteristics of a storageengine, and meanwhile, new characteristics of a modern processor are used for accelerating query.

Embodiments of the invention disclose a method and a device for monitoring a Kubernetes cluster container. The method comprises the following steps: acquiring, by the first component and the second component, a container application in the Kubernetes cluster and monitoring data of all host nodes; storing the acquired monitoring data in the time series database; using the third component to view the resource usage and performance of the Kubernetes cluster, and displaying the monitoring curve of the container application in the cluster by querying the time series database. In the embodiments ofthe invention, a monitoring component may be set up in the Kubernetes cluster, and the monitoring data of the application in the cluster may be viewed in real time, thereby realizing the refined management of the Kubernetes cluster.

The disclosed embodiments disclose techniques for managing cloud-based storage using a time-series database. A distributed cloud data management system (DCDMS) manages objects stored in a cloud storage system. The DCDMS leverages a distributed time-series database to track objects accessed via the DCDMS. During operation, the DCDMS receives a request to access an object using a path identifier and an object identifier. The DCDMS determines from the path identifier that the request is associated with one of its supported extended capabilities, and uses the previously tracked object operations that are stored in the time-series database to determine the actual target bucket in the cloud storage system that contains the requested object; the target bucket that contains the object may be different from the bucket identified in the path identifier that is received. The object identifier is then used to access the requested object from the target bucket to service the request.

The invention discloses a method and device for real-time monitoring on a time sequence database of a system. The method comprises the following steps: firstly, establishing a RowKey of an HBase so as to add monitoring indexes; storing data into the HBase, managing metadata of the database, that is, setting a metadata table, and storing a positive relationship and a negative relationship of a unique marker of the metadata by using different row clusters; and finally, filtering the database, that is, summarizing the data at a server, and reducing the data volume transmitted to a client from the server, thereby completing the design of the whole time sequence database. Compared with the prior art, the method and device for real-time monitoring on the time sequence database of the system are capable of realizing the functions of real-time writing and inquiry of the monitoring system and solving the technical problems that real-time monitoring on various indexes in the monitoring system are low in efficiency, low in speed and poor in expandability, and can be widely applied to operation and maintenance monitoring systems of various software and hardware systems.

The invention discloses a time series data storage method and time series database applied to industrial Internet of Things. The method comprises the following steps: obtaining to-be-stored time series data, and cleaning the time series data; distributing topic of the time series data according to the source of the time series data; writing the cleaned time series data into a section correspondingto a section in kafka, and caching an collection timestamp corresponding to each part of time series data and offset information of each part of time series data in the section in the writing process; organizing and combining offset information under each action, and determining an association relationship among the topic, the collection timestamp and the offset information; and persisting the incidence relation among the topic, the collection timestamp and the offset information into an index library. According to the method and the device, the data processing complexity and the data redundancy degree caused by additionally introducing a new time sequence database can be reduced.

Materialized views are maintained for time-series databases, to enable faster analytics over large data sets and to downsample data for storage savings. These materialized views are the result of computations on existing tables, such as aggregation, approximation, sampling, filtering, statistical estimates, and more. A materialized view is created on a base table, and continuously maintained, such that as changes are made to the base table, the materialized view is correspondingly updated, either synchronously or asynchronously. The database system records metadata identifying regions of the base table that have already been materialized. Subsequently, these invalidated regions are rematerialized. Queries ensure up-to-date results by combining data from the materialized view with results calculated at query time on the base table. Invalidation records and completion thresholds are used to determine which results should use records from the materialized table and which from records from the base table. Hierarchical materializations are also maintained.

The invention discloses a query request processing method and apparatus for a time series database. The method comprises the steps of determining a query type of a query request when the query request of a user is received, wherein the query type includes quick query and slow query; if the query type is the quick query, adding the query request to a queue of a first thread pool; and if the query type if the slow query, adding the query request to a queue of a second thread pool. By applying the scheme provided by the method and the apparatus, the processing efficiency and the like of a system can be improved.

A service for storing time series data provides a data pipe for receiving time series data, a query pipe for making requests to the service, and a result pipe for receiving output from the service. Data sent to the query pipe is processed by an ingester that prepares metadata indices associated with blocks of incoming time series data and stores the blocks of data in a time series database and the indices in a separate index database. A query layer receives queries from the query pipe and uses the index database to determine which data blocks are needed to process the query, and then requests only those data blocks from the time series database. Processing of the query is performed within the time series database only on those data nodes that contain relevant data, and partial results are passed to an output layer for formation into a final query result which is sent out by the results pipe.

The invention belongs to the technical field of data mining, in particular to a method for searching for an approximate sequence of a given time sequence from a massive time sequence database. The method comprises the following steps of: adopting a tree index node representation mode; constructing indexes one by one according to an algorithm frame of the indexes; selecting an optimal strategy for node splitting; and performing query on the basis of the DSTree indexes to search for the approximate sequence of the given time sequence from the massive time sequence database. According to an indexing method, the length and dimension of an index sub-sequence are regulated according to the data distribution condition of the time sequence, and by a new index representation mode, a requirement for the provision of an upper distance bound can be met, and query efficiency can be greatly improved.

The invention relates to a supermarket fruit and vegetable optimized sales device based on data mining and preservation prediction. The device comprises a fruit and vegetable rotting degree information collecting module, a sales experience data collecting module, a time series database building module, a sales experience database building module, a data warehouse building module, a knowledge model mining module, a price model building module, a real-time price setting module and a prompting module, wherein the fruit and vegetable rotting degree information collecting module collects fruit and vegetable rotting degree information; the sales experience data collecting module collects sales experience data; the knowledge model mining module obtains knowledge models through mining; the price model building module builds a fruit and vegetable price model; the real-time price setting module sets the real-time price of fruits and vegetables; and the prompting module gives out sales promotion prompt at a fruit and vegetable rotting critical point. The supermarket fruit and vegetable optimized sales device has the advantages that on the basis of the existing sales experience, the shelf life of different kinds of fruits and vegetables in different time periods is accurately estimated, and a tiered reducing price sales strategy is automatically implemented, so the fruit and vegetable sales can be effectively promoted.

The invention provides a method for integrating a system, in particular to a method for integrating a time series database with an intelligent grid scheduling technology support system so that the time series database is integrated with the intelligent grid scheduling technology support system and the intelligent grid scheduling technology support system can realize the holographic recording of the grid operation process and the efficient data inquiry on the basis of operating stably. The method provided by the invention has the advantages that: the integration of the time series database with the intelligent grid scheduling technology support system is realized, the whole grid operation process change storage mechanism can be realized according to the characteristic of the time series database, the sampling data submitting frequency is raised to the second level from the original minute level, and advanced technical means and powerful data support are provided for the fine operation analysis of the regional grid scheduling monitoring.

The invention discloses a performance testing method and system based on integrated real-time monitoring and relates to the field of performance testing. The method comprises the following steps of importing result data of a performance testing tool into a time series database and determining a first data source; when the first data source is determined, determining a second data source; and importing the first data source and the second data source into a display unit module. The system comprises a performance testing tool data transmission module, a first data acquisition module, a second data acquisition module, a data importing module, a judging module and a positioning module. According to the method and the system, all indexes are monitored in real time, the integrated monitoring isrealized, and a moment for generating a performance problem and monitoring indexes corresponding to other graphics in the moment can be judged clearly, so that the performance problem can be accurately positioned and the performance testing efficiency is improved.

This invention relates to integration circuit technique field direct standard unit database parts clock units generating method, which comprises the following steps: unit structure defining, structure building, layout distributing, RC parameter extracting, time series database files; physical files generating and testing module, wherein, the layout and lead leg is set on the adjacent metal layer channel to make the output direction on level and crossed layers to realize the lead leg use and to prevent shielding layer on the door controlled clock unit.

The invention provides a load prediction based Docker container dynamic scheduling method. The method includes: acquiring Docker container load data by a Docker container load monitoring module; storing the load data in a time series database; enabling a Docker container load prediction module to acquire and process the load data from the time series database, analyze the load data by the aid of an ARIMA model and generate a load predicated value; finally, automatically expanding quantity of Docker containers in a Docker container cluster according to the load predicated value by a Docker container scheduling module. By application of a load prediction technique to the field of Docker container scheduling, automatic expansion of the Docker container cluster according to real-time load conditions is realized, the problem of inadequate load capacity in an operating stage due to failure of the Docker container cluster in automatic resource regulation according to load conditions is effectively solved, and usability of the Docker container cluster is improved.

The present invention provides a hash ring based time sequence database service cluster implementation method and system. The system comprises a measurement point encoding module, a measurement point transmission module and a measurement point storage module. The measurement point encoding module is used for performing hash calculation according to a keyword of a measurement point to obtain a hash value corresponding to the measurement point. A code, the hash value and a measurement point value of the measurement point are serialized by the measurement point transmission module and then are transmitted to a time sequence database service cluster. The measurement point storage module creates the hash ring based time sequence database service cluster, receives measurement point data and selects, according to the hash value of the measurement point, one server from the service cluster to store an acquired value of the measurement point. According to the present invention, the load balancing of the time sequence database service cluster is realized, hardware resources of the database cluster can be fully utilized and overall data processing performance is improved.

The invention provides an index mode based on change trends of time sequences (determined time sequences and undetermined time sequences) and first-order connectivity indexes, and the index mode has great significance to time sequence prediction, classification, data mining, knowledge discovery and the like. The index mode solves the problems of high data redundancy or low match accuracy and low index efficiency caused by the time sequence space indexes, precise query, similarity query, clustering and classification of the time sequences can be finished effectively through the index, and the time complexity and space complexity of sequence query, clustering and classification are lowered greatly. According to the index mode, firstly interval segmentation is conducted on the time sequences and time dimensions, short trend symbol sequences are generated in a mapping mode according to the change trends of the time sequences in all sections, then the first-order connectivity indexes of the section rising trend, section descending trend, section wave-crest trend, section wave-trough trend and section gentle trend are calculated for the symbol sequences, and finally a B-Tree index of a time sequence database is built by the adoption of the one-order indexes of the five trends.