2110 results about "Storage efficiency" patented technology

A technique is provided whereby a stream of data located in a first storage location is registered for migration administration and according to preset criteria, portion(s) of the stream of data that are suited to another storage location are migrated to maximize system storage efficiencies. The file system may have use of facilities that enable the monitoring of files / streams that have been registered for migration administration or files / streams may be polled according to preset criteria. If the stream of data has portions to migrate to another storage location, the hierarchical storage management (HSM) system migrates the data to the other storage location, such as to long term or off-line storage, and preserves the data relationships of the stream via metadata. The technique allows for multiple volumes to be spanned by the migration target location, and multiple file systems may service the source and target.

Digital data records are stored on a direct access storage device (DASD) in emulation of sequential-access media, the efficiency of storage and access of the data being improved by using a volume trailer containing various statistics about the records. Data is preferably stored in records, logically assembled into record groups. Interspersed with the records, there may be one or more marker codes, which function like tape marks among the various data records. The volume trailer contains pointers to each record group, record counts for each group, marker code counts for each group, and may also include fields showing whether records in a given record group are of equal length. The volume trailer may also include a volume trailer "tail", including a byte count for the entire volume trailer, and a unique sequence identifying the volume trailer. Statistics contained in the volume trailer enable substantially more efficient access of the data by a DASD. For example, record groupings, group counts, and / or marker code counts may be employed to perform operations such as locate, forward space file, and backward space file.

The present invention is to provide a system for backing up cache memory in a double backup server structure, which comprises a first backup server having a first SAS controller; a second backup server having a second SAS controller; and a SAS channel for connecting said first and second SAS controllers; wherein said first SAS controller and said second SAS controller each having a sending end initiator defining a cache mirror command. When said first backup server received a write command and write data through a Fiber Channel or iSCSI channel into a first cache memory, said first SAS controller also establishes a first Command Descriptor Block (CDB) with an operation code of a cache mirror command as well as a first data header for the cache data. Said first CDB and first data header are subsequently transmitted through said SAS channel to said second SAS controller and said second backup server writes said data into a second cache memory of its own. Accordingly, with the higher bandwidth facilitated by the SAS channel, considerable amount of cache data can be processes in high speed to improve the storage efficiency in a double backup server structure.

A storage area network can include a storage virtualization entity—intelligent storage application resource (iSAR)—either as a separate device in the fabric, or as an integrated module in one or more switches within the fabric. All I / O operations can be re-directed to iSAR for processing. iSAR can segment virtual storage and physical storage into units, where each unit of the virtual storage is mapped to a single unit in physical storage. Data associated with incoming I / O operation can be compressed before being stored in physical storage. iSAR includes overflow reserve storage at the block, sub-page and page level to accommodate changes in compressed data size on subsequent I / O operations. These measures can improve I / O performance and reduce fragmentation. iSAR can also employ deduplication of incoming data stored on physical storage to improve storage efficiency.

Mechanisms are provided for efficiently improving a dictionary used for data deduplication. Dictionaries are used to hold hash key and location pairs for deduplicated data. Strong hash keys prevent collisions but weak hash keys are more computation and storage efficient. Mechanisms are provided to use both a weak hash key and a strong hash key. Weak hash keys and corresponding location pairs are stored in an improved dictionary while strong hash keys are maintained with the deduplicated data itself. The need for having uniqueness from a strong hash function is balanced with the deduplication dictionary space savings from a weak hash function.

A method and apparatus are disclosed for increasing virtual storage capacity in on-demand storage systems. The method utilizes data compression to selectively compress data stored in a storage resource to reduce the utilization of physical storage space whenever such physical resources have been over committed and the demand for physical storage exceeds its availability. In one exemplary embodiment, the utilization of the capacity of a shared storage resource is monitored and data is selected for compression based on the utilization. The compression of the selected data is triggered in response to the monitoring results. In addition, policies and rules are defined that determine which data is selected for compression. For example, the selection of data may be based on one or more of the following: a degree of utilization of said capacity of said shared storage resource, a volume size of said data, an indicator of compressibility of said data, a frequency of use of said data, a manual selection of said data, and a predefined priority of said data. The disclosed methods improve the operation of virtual allocation by further enhancing the availability of physical space through data compression. Virtual allocation and block-based data compression techniques are utilized to improve storage efficiency with a minimal risk to system availability and reliability and with a minimal impact to performance (access time and latency).

The invention is high performance Syslog log processing and storing method, and the steps are: (1) receiving log: log sever software receives the syslog data message by separated thread running method via bound UDP port (514), and the data message is generated into data class of syslog, and the data class of syslog is written in log buffer; (2) log buffer: the log buffer takes charge of storing the received syslog log data temporarily; (3) log normalization processing: detects the head of log buffer by separated thread, if the log buffer has new data, and the data will be extracted from buffer queue, and log will execute normalization processing according to field description information of log normalization configuring file, and call the log storing module to store the log as uniform format; (4) log storage: using the storing thread pool to write said normalization log on log storage file, every thread takes charge of writing log data in one file, and several threads can write the logs in different directories and discs, and improve the storing efficiency. The technical schedule adopts asynchronies log processing and intercurrent log storage technique, and two taches can inherit each other and coordinates the problems of log amount and log inercurrent amount, and use time to change time, and improve the log processing ability greatly.

In an aggregated file system, a file may begin with a set of stripe fragments all in the RAID-5 scheme in order to take advantage of the RAID-5 scheme's storage efficiency. After that, when one of the fragments is accessed by a file switch, it will be duplicated into the data mirroring scheme. The file's corresponding metadata server maintains a data structure, e.g., a bitmap, indicating which fragments have been duplicated into the data mirroring scheme. In other words, the file, at this moment, exists in a hybrid scheme. A file consolidator running on the metadata server is triggered at a predefined time to copy the fragments from the data mirroring scheme back to the RAID-5 scheme, This file consolidator also updates the bitmap to reflect the changes to the file's scheme change. This hybrid scheme is expected to increase the I / O capacity of the conventional RAID-5 scheme and the storage usage of the conventional mirroring scheme.

An application server disk image generating apparatus and method comprising a basic disk image generating component for generating basic disk images for basic programs used by the server; an incremental disk image generating component for generating incremental disk images including heritage relationship for other respective applications based on the basic programs used by the server; and an image profile generating means for generating an image profile for each of the basic disk images and the incremental disk images. By using the apparatus, the storage consumption can be decreased greatly and the storage efficiency can be improved. The invention also discloses an application server disk image management and distribution system to which the application server disk image generating apparatus is applied, and a system for provisioning the application server using disk images.

The invention provides a similar web page duplicate-removing system based on a parallel programming mode, comprises a web page content pre-processing module, a web page eigenvector extracting module,a web page feature fingerprint calculation module, a web page fingerprint on-line duplicate-removing module, a web page fingerprint distributed batch duplicate-removing module and a computing platformbased on specific distribution. The system can complete links of carrying out unified conversion of text content encoding, standardization of document structure, web page noise content abortion, thematic content analysis and identification of web pages, lexical segmentation of continuous text content, and the like on the web pages obtained by crawling of web crawlers, thereby forming eigenvectorswhich can present the web pages. Relative algorithms can be used to obtain web page fingerprints which present web page characteristics aiming at the vector. The system provided by the invention accurately and fast detects fully complete repetition or approximate repetition of the web page contents caused by site mirroring, web document transshipment, and the like on the condition of massive amount of data of Internet and completes corresponding repetition-removing works, thereby enhancing the storage efficiency of search engines and bringing better use experience for the search engines.

A technique is provided whereby a stream of data located in a first storage location is registered for migration administration and according to preset criteria, portion(s) of the stream of data that are suited to another storage location are migrated to maximize system storage efficiencies. The file system may have use of facilities that enable the monitoring of files / streams that have been registered for migration administration or files / streams may be polled according to preset criteria. If the stream of data has portions to migrate to another storage location, the hierarchical storage management (HSM) system migrates the data to the other storage location, such as to long term or off-line storage, and preserves the data relationships of the stream via metadata. The technique allows for multiple volumes to be spanned by the migration target location, and multiple file systems may service the source and target.

The invention discloses a Hadoop-based mass classifiable small file association storage method, which mainly solves the problem of low access efficiency of classifiable small files. The method comprises a NameNode global index management technology and a file clustering technology. Independent small files belonging to a certain class are subjected to file clustering and global index management, so that the utilization rate of an internal memory is greatly improved and the maximum file number supported by unit internal memory is improved. The method comprises the following steps of: (1) clustering small files belonging to a certain class into a file called a logic unit; and (2) establishing a global index for each small file stored in a NameNode internal memory. The file clustering technology is used for improving the storage efficiency of the classifiable small files, and the NameNode global index management technology is used for managing the clustered small files. By the technologies, the storage efficiency of the mass classifiable small files is improved. The invention is suitable for storing and managing the classifiable small files under general scenes.

The invention discloses a mass non-independent small file associated storage method based on Hadoop. The method is mainly used for solving the problem of low mass non-independent small file access and reading efficiency and aims at a plurality of small files, namely non-independent small files obtained by cutting a big file. The method is characterized by comprising the following steps: (1) merging all the small files of the big file into one file which is named merged file; (2) establishing a local index for each merged file, and storing a local index file and a file entity onto a Data Node of a Hadoop system while updating; and (3) when the non-independent small files are read, improving the file reading efficiency by adopting metadata cache, local index file pre-fetching and associated file pre-fetching. By utilizing the method provided by the invention, the efficiency of the existing Hadoop system for storing and reading small files is improved. The method is suitable for the storage and management of the mass non-independent small files in universal scenes.

An automatic warehouse 1 includes a plurality of shelves 50 and a conveyance device 20. The plurality of shelves 50 are arranged in a horizontal direction and in a single level or in a plurality of levels and adapted to store storage objects. The conveyance device 20 is adapted to transfer the storage objects between the same and each of the plurality of shelves 50. All of the plurality of shelves 50 are implemented by a plurality of shelves 50 provided in a plurality of carriages 60 capable of moving between the inside of the automatic warehouse and the outside of the automatic warehouse. By use of the plurality of carriages 60 which also function as storage shelves, there can be provided an automatic warehouse having high degree of freedom and good storage efficiency, enabling efficient use of factory space, and being capable of coping with cell production.

The invention provides a composite phase-change thermal storage material. A porous material with high thermal conductivity is used as a supporting framework, and low-melting-point metal or low-melting-point metal with nano-particles is distributed in pores of the porous material, wherein melting point or solidus temperature of the low-melting-point metal is less than or equal to 80 DEG C; and thermal conductivity of the porous material is within 40-400 W / (m.K). The material provided by the invention has high equivalent thermal conductivity and high storage energy density; there is a large contact area between the liquid metal and the porous material; and the material has a wide application temperature range, has good fixability, stable physico-chemical property and good reversibility; and the problem of decreasing heat storage efficiency after multiple times of heat adsorption and release cycles is avoided.

Apparatuses, systems, and methods for optimal group key (OGK) management that may achieve non-colluding and / or the storage-communication optimality are disclosed. In some embodiments, a group controller (GC) is responsible for key generation and distribution and the group data are encrypted by a group key. When joining the group, in some embodiments, each group member (GM) is assigned a unique n-bit ID and a set of secrets, in which each bit is one-to-one mapped to a unique secret. Whenever GMs are revoked from the group, in some embodiments, the GC will multicast an encrypted key-update message. Only the remaining GMs may be able to recover the message and update GK as well as their private keys. The disclosed OGK scheme can achieve storage-communication optimality with constant message size and immune to collusion attack and also may outperform existing group key management schemes in terms of communication and storage efficiency.

The invention relates to a safety metadata management method based on integrality checking, belonging to the field of storage safety. The method is characterized in that a self encryption technology for file data is used at a client to encrypt the file data, thereby improving the storage efficiency; a 64-heap hash tree is used at a safety metadata management server end to maintain the hash value of a file block, thereby providing integrality protection on the hash value of the file data block and realizing the high-efficiency verification on the user's access to file and the high-efficiency data updating; and simultaneously a root hash virtual linked list technology is used, thereby ensuring the integrality of root hash values of multiple 64-heap hash trees belonging to one file and supporting the user's concurrent access to the file to a certain extent.

A method and apparatus is provided for monitoring the operation of a NOx storage catalytic converter in the exhaust stream of an internal combustion engine. The NOx content of the gas stream upstream and downstream of the catalytic converter is measured and used to determine storage efficiency. This is compared to a threshold value, and if the threshold value is not met a regeneration process is performed. The storage value is measured after regeneration to determine if the catalytic converter is faulty.

The invention discloses a realization method of a memory database on a home gateway. The method of the invention comprises the steps: 1. the storage structure and queue of data sheets and records in the memory database are initialized; 2. on the basis of the establishment of the storage structure of the memory database, the business data in a business database are uploaded to the memory database on the home gateway, while before the business data are uploaded, the business data are filtered; 3. data synchronization: when the data in business database are changed, the given data in the memory database are backed up; data synchronization is carried out by adopting a synchronous module, and if the synchronization succeeds, the backup data are cleared; 4. data access: a unified access interface of the memory database is provided for inquiring, adding, deleting and modifying, etc. Compared with the realization method of the existing memory database, the realization method of the invention requires few resources, has high data storage efficiency and is especially suitable for the business application that is based on a home gateway and has high requirement for data real time.

An improved mobile apparatus for sterilizing surgical trays serves as a self-contained autoclave, allowing sterilization of the interior of the apparatus and its contents. A method of sterilization using the apparatus is presented as well. By means of a dedicated transfer and storage system, the apparatus, integrable with respect to a transfer cart and a lift device, may be moved easily between a location of sterilization, a storage area, and an operating room, and more than one such apparatus may be stacked vertically for storage to enhance storage efficiency.

The present invention discloses a distribution type of audience-choose-broadcasting video system and its data storing and accessing method. Said storing method includes the following steps: dividing a data file into data blocks in a predetermined manner; forming a data checking block for calculating the data blocks having predetermined number; storing the said data blocks and the data checking block into a plurality of predetermined buffer-server in turn in a predetermined manner; and creating corresponding index information for storing data blocks in each said buffer-server. The technical solution of the invention can improve storing efficiency and save a lot of storing space; and increase the safety of data because adopting the distribution type of memory and the data are distributed among a plurality of buffer-server.

A computer implemented device is described that is adapted for improving memory efficiency for conducting machine learning on multi-dimensional vectors stored as specially configured data structures. The device includes network interfaces adapted for receiving data sets and communicating with data harvesting applications, as well as data storage and memory for storing the configured data structures. The multi-dimensional vectors and the system provide a data architecture that processes disparate data sets to programmatically extract features transformed from raw information, the extracted features stored in the form of data values suitable for conducting data approximations and neural network processing.

The invention relates to the field of a video coding technology, and discloses a region of interest (ROI) video coding method and apparatus based on object detection. The video coding method comprises the following steps: performing the object detection on an original image, generating object information, and obtaining an object area and confidence; converting the object area into an ROI suitable for coding; according to coding quality grades and the confidence, calculating coding quantification parameters; and according to the corresponding coding quantification parameters, performing quantification coding on the ROI and a non-ROI. According to the invention, the ROI, the non-ROI and the corresponding confidence are obtained through the object detection, and the corresponding quantification parameters are selected for performing the quantification coding on the ROI and the non-ROI, such that the coding quality and stability of the ROI can be effectively ensured, at the same time under the condition that the coding quality of the ROI is not reduced, the compression code stream of a whole image can be decreased, and the video transmission and storage efficiency is improved.

The present application discloses a distributed storage system. The distributed storage system comprises an agent module, a metadata service module and a plurality of storage service modules, wherein each storage service module manages at least one storage node; and the metadata service module is used for maintaining a global first level mapping table, and the storage service module is used for maintaining a second level mapping table. As adoption of a two-level metadata organization, when data is written, if same data exists, actual writing is not carried out, thereby reducing the data writing operation and fulfilling the aim of deleting the global duplicated data. The present application further discloses a corresponding data read-write method. The application scheme can achieve the following technical effects: achieving an online global duplicate data delete in the distributed storage system, thereby reducing the actual storage space; and when writing the data, as the duplicate data does not need the actual writing operation, thereby saving the data communication and bandwidth of the application systems and the storage systems, and improving the storage efficiency.

The invention discloses a method and a system for layer-management of a storage system. The method includes the following steps that according to medium types of storage media in the storage system, each storage medium is divided into at least two storage layers; according to data types of data objects, the data objects are stored in corresponding storage layers. By means of layer data storage, the method and the system are applied to a distributed file system, automatic layer storage of data and automatic migration of data are supported, data storage cost can be greatly reduced, and storage efficiency is improved. Meanwhile, data can be managed reasonably, and the method and the system have the advantages of good controllability and expandability.

A recovery enabling system for storage arrays is a high distance generalization of RAID-5 with optimal update complexity and near optimal storage efficiency. The recovery enabling system utilizes presets, data cells with known values that initialize the reconstruction process. The presets allow resolution of parity equations to reconstruct data when failures occur. In one embodiment, additional copies of the layout of the recovery enabling system are packed onto the same disks to minimize the effect of presets on storage efficiency without destroying the clean geometric construction of the recovery enabling system. The recovery enabling system has efficient XOR-based encoding, recovery, and updating algorithms for arbitrarily large distances, making the recovery enabling system an ideal candidate when storage-efficient reliable codes are required.

The method includes: 1, Divide the media file into several fragments. Each fragment forms a slice with fixed size. Each slice at least includes one slice head and one data area. The head at least includes the slice number and the media label corresponding to this slice. The data area is for storing audio / video frame of the fragment corresponding to the slice according to the time stamp sequence; 2, Store evenly the formed slices in step 1 into several media units and build the storage record. This invention is mainly applied in grouped VOD system. It enhances storage efficiency, retrenches tediously load-balance calculation and increases the system expansibility.

The invention discloses an efficient data encryption, updating and access control method for cloud storage, which belongs to the technical field of network information safety and computer software. The method can realize quick block encryption replacement for plaintext data, so that data confidentiality is guaranteed while abuse of user data by a cloud server can be prevented. The method can realize efficient updating for ciphertext data without re-encrypting and uploading a whole data file, so that system transmission and storage efficiency is improved. The method can realize dual permission control for the data file, ciphertext access rights are formulated by data owners but executed by the cloud server, and secret key materials for decrypting ciphertexts are controlled by the data owners, so that data file control is enhanced, and permission revocation can be flexibly realized. Encryption of the plaintext data and processing of the ciphertext data can be realized by the method at low cost, duel replacement for the plaintext data enables a data applicant to obtain completely different ciphertext results in each time of application and prevents the cloud server from abusing the user data, accordingly data confidentiality is further enhanced, and the method has bright application prospect in data cloud storage.