132 results about "Redo log" patented technology

A server-based desktop-virtual machines architecture may be extended to a client machine. In one embodiment, a user desktop is remotely accessed from a client system. The remote desktop is generated by a first virtual machine running on a server system, which may comprise one or more server computers. During execution of the first virtual machine, writes to a corresponding virtual disk are directed to a delta disk file or redo log. A copy of the virtual disk is created on the client system. When a user decides to “check out” his or her desktop, the first virtual machine is terminated (if it is running) and a copy of the delta disk is created on the client system. Once the delta disk is present on the client system, a second virtual machine can be started on the client system using the virtual disk and delta disk to provide local access to the user's desktop at the client system. This allows the user to then access his or her desktop without being connected to a network.

One embodiment is a method for providing incremental backups for a source computing machine, the method including: (a) creating a first backup snapshot including a virtual machine (VM) snapshot of an initial copy of a source computing machine volume, wherein said VM snapshot includes a timestamp and a first redo log file; (b) reconfiguring and customizing said first backup snapshot to create a first bootable VM, writing changes associated with said reconfiguring and customizing into said first redo log file, then creating a first bootable snapshot including a VM snapshot of said first bootable VM, wherein said VM snapshot of said first bootable VM includes a timestamp for said first bootable VM and a redo log file; (c) performing an incremental update of said first backup snapshot or a subsequent backup snapshot, then creating a subsequent backup snapshot including a VM snapshot of said incremental update, wherein said VM snapshot of said incremental update includes a timestamp for said incremental update and a redo log file; and (d) reconfiguring and customizing said subsequent backup snapshot to create a subsequent bootable VM, then creating a subsequent bootable snapshot including a VM snapshot of said subsequent bootable VM, wherein said VM snapshot of said subsequent bootable VM includes a timestamp for said subsequent bootable VM and a redo log file.

A method for applying changes to a standby system is described in which multiple apply tasks or “slaves” are scheduled to changes from redo logs, especially those changes in large transactions, in parallel and as soon as they are encountered in the logs. To foster data convergence, a partial ordering may be imposed on the transactions so that those transactions that actually depend on others are forced to wait on the transactions they are dependent upon commit. To foster read consistency, synchronization points may be periodically established, at which points a read consistent state of the standby database system can be obtained-all without adversely impact the throughput of the system.

A parallel logging and recovery scheme for highly available main-memory database systems is presented. A preferred embodiment called parallel redo-only logging (“PROL”) combines physical logging and selective replay of redo-only log records. During physical logging, log records are generated with an update sequence number representing the sequence of database update. The log records are replayed selectively during recovery based on the update sequence number. Since the order of replaying log records doesn't matter in physical logging, PROL makes parallel operations possible. Since the physical logging does not depend on the state of the object to which the log records are applied, the present invention also makes it easy to construct a log-based hot standby system.

A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

The present invention is constituted so as to enable the recovery of a database even when a log comprising a DB update history is not created. When executing a transaction in the logless mode, a logless-start log is created and written to a primary log VOL 55A at the start of this transaction, and this logless-start log is also written to a secondary log VOL 55B. Thereafter, batch processing is carried out for a DB of a primary DB VOL 51A, and while this transaction is being executed, the pair-split state of the DB VOL is cancelled either prior to the start or after the end of the batch processing, and if the difference between the DB VOLs is eliminated subsequent to the end of batch processing, a logless-end log is created and written to the primary log VOL 55A, and this log is also written to the secondary log VOL 55B.

Techniques for automatically changing the mode used in a primary database system to transport redo to a standby database system in response to changing workload and network conditions. The techniques are implemented in a database system that has a constraining redo transport mode that can potentially constrain the rate at which the primary database system can process transactions and a nonconstraining redo transport mode which does not constrain the primary but has a higher probability of redo loss than the constraining redo transport mode. The techniques use the constraining redo transport mode as a measuring transport mode to determine whether a switch from one mode to the other is desirable either to increase the throughput of the primary database system or to decrease the probability of the loss of redo data.

Systems and methods for logging and recovering updates to data structures in the event of failure of an information management system are provided. In exemplary implementations, methods for implementing an efficient redo log for a data structure that is concurrently accessed by multiple clients is provided. The data structure is implemented in two layers: the data structure algorithm layer which sits atop an allocator that provides distributed, persistent, and replicated storage allocation. Both the B-link tree algorithm layer and the allocator use the service of the logging mechanism to implement fault-tolerance and atomicity guarantees. The present invention uses a single log and allows periodic truncation of that log for space efficiency.

An improved method, mechanism, and system for implementing, generating, and maintaining records, such as redo records and redo logs in a database system, are disclosed. Multiple sets of records may be created and combined into a partially ordered (or non-ordered) group of records, which are later collectively ordered or sorted as needed to create an fully ordered set of records. With respect to a database system, redo generation bottleneck is minimized by providing multiple in-memory redo buffers that are available to hold redo records generated by multiple threads of execution. When the in-memory redo buffers are written to a persistent storage medium, no specific ordering needs to be specified with respect to the redo records from the different in-memory redo buffers. While the collective group of records may not be ordered, the written-out redo records may be partially ordered based upon the ordered redo records from within individual in-memory redo buffers. At recovery, ordering and / or merging of redo records may occur to satisfy database consistency requirements.

A method and system that annotates a redo log to provide information concerning the execution of a procedure at a primary database. The annotations include entry and exit markers that indicate the beginning and the end of the execution along with any arguments passed to the procedure, and whether the execution of the procedure was successful. At the standby database, these markers are used to create a logical transaction associated with the procedure. The operations performed by the procedure are grouped into individual transactions, and these individual transactions are grouped as belonging to the logical transaction. If the execution of the procedure was successful at the primary database, then the individual transactions are discarded, and the logical transaction is applied by executing the procedure at the standby database. If the execution of the procedure failed at the primary database, then the individual transactions and the logical transaction are discarded.

A method for providing data integrity of a database system. A standby database is selected to replace a failed primary database. At least one redo log file having at least one data transaction is selected to be applied to the selected standby database. The selected standby database is updated with the data transaction from the selected redo log file if an identification code in the redo log file matches an identification code in the standby database.

Techniques are provided for detecting lost writes so that data corruption can be avoided. According to one technique, lost writes are detected by comparing version numbers that are indicated in redo log entries with version numbers that are indicated in corresponding blocks in a standby database. According to one technique, which may be used in conjunction with the above technique, lost writes are detected by logging read operations that occur relative to a primary database, and comparing version numbers that are indicated in read log entries with version numbers that are indicated in corresponding blocks in a standby database. According to one technique, lost writes are detected by comparing, in response to read operations, (a) version numbers that are stored in a persistent in-memory cache with (b) version numbers that are indicated in corresponding blocks that are read from a primary database.

A method for performing a system level backup of a log-ahead database management system (DBMS) without suspending updates by application programs is described. The DBMS comprises a database mainline system, a backup utility and a restore utility. The data and log records are stored on separate storage volumes. Log records are written to identify objects that require special handling during the point-in-time recovery. The database engine operates normally during a backup except for suspending actions that would alter the file system catalog or write updates across a storage volume boundary; and by freezing the REDO log point in its checkpoint information. The backup utility copies the data volumes first and optionally the log volumes second while updates are allowed. The resulting inconsistencies are resolved either during a DBMS restart or during a point-in-time (PIT) recovery performed by the restore utility.

A method for providing data integrity of a database system. A standby database is selected to replace a failed primary database. At least one redo log file having at least one data transaction is selected to be applied to the selected standby database. The selected standby database is updated with the data transaction from the selected redo log file if an identification code in the redo log file matches an identification code in the standby database.

Exemplary methods, apparatuses, and systems create a parent snapshot of the parent virtual disk in response to receiving a request to clone a parent virtual machine. A clone virtual machine is created with a clone redo log file and clone virtual disk. Copying of the parent virtual disk to the clone virtual disk is initiated. A first active link from the clone redo log file to the parent snapshot is created to provide the clone virtual machine access to the parent snapshot. In response to determining that the copying of the parent virtual disk to the clone virtual disk is complete, the first active link is removed and a second active link is created from the clone redo log file to the clone virtual disk or to a snapshot of the clone virtual disk.

The embodiment of the invention discloses a data synchronization method and a relevant device. The method comprises the steps that a main node acquires redoing logs of a changed page during transaction submission, and a page identifier of the changed page, a current transaction snapshot and log serial numbers are recorded; the main node writes the redoing logs into a storage node, and the redoinglogs are used for the storage node to fix data in the changed page; and the main node sends an invalidity instruction message to a slave node, wherein the invalidity instruction message contains the page identifier, the current transaction snapshot and the log serial numbers and is used for instructing a data page corresponding to an invalid page identifier of the slave node, determining version information of a data page visible to the message according to the current transaction snapshot and acquiring a data page successfully fixed through a target redoing log from the storage node, and theserial number of the target redoing log is greater than or equal to the log serial number corresponding to the version information. By the adoption of the embodiment, data synchronization efficiency and accuracy are improved.

Techniques for reducing commit latency in a database system having a primary database system and a standby database system that is receiving a stream of redo data items from the primary. The standby sends an acknowledgment for a received item of redo data before the standby writes the redo data item to a redo log for the stream. When a no more redo event occurs in the standby, the standby sets a “no data lost flag” in the redo log if the stream of redo data items has no gaps and all of the redo data items received in the standby have been written to the redo log. The database system may operate in a first mode in which an acknowledgment is sent as just described and a second mode in which an acknowledgment is sent after the redo data item has been written to the redo log.

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO / REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.

Redo logs are used to facilitate efficient cloning of virtual machines. When a virtual machine with a virtual hard disk is to be cloned, two redo logs are created, both of which are linked to the virtual hard disk. The virtual machine being cloned is then linked to one redo log, and a newly created virtual machine is linked to the other. Each time an additional virtual machine is created, two new redo logs are created and linked to the end of the disk chain. The parent and newly created virtual machine are each linked to one of the new redo logs.

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO / REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.

The invention discloses a method and device for redoing logs of database records, and belongs to the technical field of computers. The method comprises the following steps of receiving a log record request, wherein the log record request carries a transaction identifier and log information; judging the log record request to determine whether the log record request is the first log record request corresponding to the transaction identifier; if the log record request is the first log record request, selecting part of buffer space from a redone buffer, establishing an transaction buffer block corresponding to the transaction identifier, and recording the log information and the corresponding global sequence information into the transaction buffer block; if the log record request is not the first log record request, determining the transaction buffer block currently used in the established transaction buffer block corresponding to the transaction identifier, and recording the log information and the corresponding global sequence information into the transaction buffer block; and during occurrence of the preset trigger transaction, sustaining the log information in the redone buffer. The method and the device have the advantage that the processing speed of a database system can be increased.

The invention provides a database synchronization method and a database synchronization system. The database synchronization method comprises the following steps that an online redo log of a source database is obtained, and a corresponding capture thread is created; the online redo log is analyzed; the capture threads are switched according to the analysis result, and incremental data in the source database is obtained through the switched capture thread; the captured incremental data is transmitted to a corresponding transmit queue; the incremental data in the transmit queue is subjected to adaptation and is then transmitted to a target database for synchronization. With the database synchronization method and the database synchronization system, the database synchronization speed can be improved.

The invention discloses a distributed parallel processing database system and a data processing method thereof. Reliable data support is provided for data recovery by formulating a data fragment distribution rule, a redundant backup rule and a redo log mechanism. And according to the data operation type, log recovery or inter-node consistency recovery is redone to perform data automatic recovery.Distributed deployment is supported, automatic data recovery is achieved, and a stable storage computing service is provided.

The invention discloses a database processing method, device and system, and the method comprises the steps: enabling a main node to write a redo log record generated by a modification operation or awrite-in operation into a shared storage node when the main node executes the modification operation or the write-in operation of a data record, and updating a corresponding dirty data page to the shared storage node; and enabling the replica node to read the redo log record from the shared storage node into a memory, so that a data page can be reconstructed through the redo log record when needed. The deployment cost and the master-slave delay of the database for realizing read-write separation can be reduced.

The location of appropriate undo data necessary for reversing an effect of an operation performed at a computer system is identified during a forward pass through a log. A cycle count can be used to determine if appropriate undo data is stored in an object or is stored in auxiliary storage. If appropriate undo data is stored in an object and the undo data would be potentially lost by processing a redo log entry to modify the contents of the object, the appropriate undo data is copied to auxiliary storage. In some embodiments, a secondary computer system receives log entries from a primary computer system. The secondary computer system can identify undo data necessary to reverse an effect of an operation performed at the secondary computer system from received log entries. Identified undo data can be copied auxiliary storage at the secondary computer system.

Techniques are provided for detecting lost writes so that data corruption can be avoided. According to one technique, lost writes are detected by comparing version numbers that are indicated in redo log entries with version numbers that are indicated in corresponding blocks in a standby database. According to one technique, which may be used in conjunction with the above technique, lost writes are detected by logging read operations that occur relative to a primary database, and comparing version numbers that are indicated in read log entries with version numbers that are indicated in corresponding blocks in a standby database. According to one technique, lost writes are detected by comparing, in response to read operations, (a) version numbers that are stored in a persistent in-memory cache with (b) version numbers that are indicated in corresponding blocks that are read from a primary database.

The invention discloses a primary and backup copying method of a memory database and a memory database system. A mode of combining asynchrony and semi-synchronicity is used. Compared with a conventional semi-synchronous copying mode, the method provided by the invention has the advantages that according to the semi-synchronicity, the subsequent processing can be performed without waiting for the receiving confirmation of a backup library host on a REDO log sent by a primary library; the processing efficiency cannot be influenced; in addition, the asynchronous copying and the semi-synchronous copying are mutually matched for jointly ensuring the consistency of data in a primary library and a backup library; under the condition that the main library is abnormal, the backup library can take over the function of the primary library; and on the precise of ensuring the data security, the efficient processing advantages of the memory database are sufficiently realized.

The invention discloses a recorded data updating method and device. The recorded data updating method comprises the steps that each of a plurality of update operations of one record is updated for a request, and recorded data of the record buffered in the update operation are updated recorded data; on the basis of the updated recorded data corresponding to the buffered update operations, redo logs of the record corresponding to the multiple update operations are written into persistent equipment at a time; when the redo logs are written successfully, the recorded data of the record in a database are updated by means of the updated recorded data corresponding to the multiple buffered update operations. By the adoption of the scheme, the concurrent recorded data updating processing efficiency is improved.

The embodiment of the invention discloses a log replay method and device, a server and a storage medium. The method is applied to a main database. The method comprises: if it is determined that a current database instance modifies a target data page for the first time, generating a redo log, and the redo log comprising a page synchronization redo log and a conventional redo log; and sending the redo log to the standby database to enable the standby database to perform log replay according to the log type of the redo log. According to the embodiment of the invention, when the main database generates the redo log, a new redo log type of the page synchronization redo log is added, so that the standby database can process the redo log except the page synchronization redo log in parallel according to the log type without additional logic judgment, and the replay efficiency and performance of the standby database are greatly improved.