5200 results about "Data transformation" patented technology

A method and apparatus is disclosed for creating data transformation routines for transforming data from a source format to a target format. The routines can be used in conjunction with dynamic binary translators, which enable binaries of a source platform to execute on a target platform without recompilation. This is achieved by runtime (on-the-fly) translation of source machine instructions into equivalent target machine instructions. Typically dynamic binary translators are used for migrating from an older platform to a newer one, implementing complex instruction set architectures, speeding up simulators and in profiling tools. The transformation routines are also used for translating data in databases from one format to another, for example, where the data format has been modified in an upgraded system. In all these applications, the speed and accuracy of translation need to be kept low to ensure that the overhead incurred does not outweigh the advantages of translation.

What is described is a method and a system for data transformation by using predefined mapping rules. A transformation between a source data object and a target data object is defined and an initial mapping of elements from the source data object to the target data object is created. A predefined mapping rule is applied as a subsequent mapping between the source data object and the target data object to adjust the transformation. The mapping from the source data object to the target data object is displayed via a graphical user interface.

A software application for enabling creation and execution of an automated browser navigation sequence is provided. The software application comprises a session recording module for recording parameters associated with a manual navigation sequence, a file creation module for converting data of a manual session into data comprising an executable sequence of instructions for conducting an automated navigation sequence, and an application-program-interface module for integrating a functional capability with the automated navigation sequence. The automated navigation sequence is characterized in that a completely automated browser-navigation sequence performed by the browser application is enabled through execution of the executable instruction sequence created from the recorded parameters of the manual navigation sequence.

Real time data integration jobs are deployed in a services oriented architecture as services that allow multiple service bindings. In one aspect, a method includes providing a code module executing a data integration job and a registry of services for storing the code module. When the code module is identified in the registry, access is provided to the code module in the registry of services, in real time wherein the code module may be accessed by more than one type of service binding such that different applications can utilize the real-time integration service using different protocols to invoke the real-time integration service. At least one of the types of service bindings may be an EJB binding, a SOAP binding, a JMS binding, and a web service binding. The data integration job may include an extraction job, a data transformation job, a loading job, a metadata management job, and a data quality job.

One aspect of the present invention includes a configuration of a storage management system that enables the performance of deduplication activities at both the client (source) and at the server (target) locations. The location of deduplication operations can then be optimized based on system conditions or predefined policies. In one embodiment, seamless switching of deduplication activities between the client and the server is enabled by utilizing uniform deduplication process algorithms and accessing the same deduplication index (containing information on the hashed data chunks). Additionally, any data transformations on the chunks are performed subsequent to identification of the data chunks. Accordingly, with use of this storage configuration, the storage system can find and utilize matching chunks generated with either client- or server-side deduplication.

Prediction-based compression engines are spoon-fed with sequentially efficiently compressible (SEC) streams of input data that make it possible for the compression engines to more efficiently compress or otherwise compact the incoming data than would be possible with streams of input data accepted on a TV-raster scan basis. Various techniques are disclosed for intentionally forming SEC input data streams. Among these are the tight packing of alike files or fragments into concatenation suitcases and the decomposition of files into substantially predictably consistent (SPC) fragments or segments that are routed to different suitcases according to their type. In a graphics-directed embodiment, image frames are partitioned into segment areas that are internally SPC and multidirectional walks (i.e., U-turning walks) are defined in the segment areas where these defined walks are traced during compression and also during decompression. A variety of pre-compression data transformation methods are disclosed for causing apparently random data sequences to appear more compressibly alike to each other. The methods are usable in systems that permit substantially longer times for data compaction operations than for data decompaction operations.

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction / transform / load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

The present disclosure relates generally to a data enrichment service that extracts, repairs, and enriches datasets, resulting in more precise entity resolution and correlation for purposes of subsequent indexing and clustering. As the data enrichment service can include a visual recommendation engine and language for performing large-scale data preparation, repair, and enrichment of heterogeneous datasets. This enables the user to select and see how the recommended enrichments (e.g., transformations and repairs) will affect the user's data and make adjustments as needed. The data enrichment service can receive feedback from users through a user interface and can filter recommendations based on the user feedback.

Prediction-based compression engines are spoon-fed with sequentially efficiently compressible (SEC) streams of input data that make it possible for the compression engines to more efficiently compress or otherwise compact the incoming data than would be possible with streams of input data accepted on a TV-raster scan basis. Various techniques are disclosed for intentionally forming SEC input data streams. Among these are the tight packing of alike files or fragments into concatenation suitcases and the decomposition of files into substantially predictably consistent (SPC) fragments or segments that are routed to different suitcases according to their type. In a graphics-directed embodiment, image frames are partitioned into segment areas that are internally SPC and multidirectional walks (i.e., U-turning walks) are defined in the segment areas where these defined walks are traced during compression and also during decompression. A variety of pre-compression data transformation methods are disclosed for causing apparently random data sequences to appear more compressibly alike to each other. The methods are usable in systems that permit substantially longer times for data compaction operations than for data decompaction operations.

Systems and methods which provide translation and / or transformation of messaging data, such as that collected by a distributed management method, for collection, consolidation, processing, and / or analysis are shown. Preferred embodiments of the present invention provide data translation schema to manipulate raw data stored in a first database or databases, such as a staging database, to store useful information in appropriate locations of a second database or databases, such as a data warehouse. Preferably, for each data source monitored in a messaging environment, collected raw messaging data is parsed into appropriate positions of a staging database. Thereafter, data transforms of the present invention preferably manipulate and / or process data of the staging database and provide useful information for storage in a data warehouse.

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction / transform / load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

The invention provides an intelligent cloud gateway of the industrial IoT and a communication method of the gateway. The gateway comprises a data collection module, a protocol analysis and conversionmodule and a cloud transmission module; the data collection module obtains industrial data collected by network equipment in the heterogeneous network; the protocol analysis and conversion module analyzes protocols of the industrial data, converts the industrial data with the different communication protocols into OPC UA standard data, and further converts the obtained OPC UA standard data into MQTT protocol data; and the cloud transmission module transmits the MQTT protocol data to a cloud platform. Thus, the industrial data with the different protocols is sent to the cloud via the single gateway, different network device access schemes are provided, and the difficulty in transmitting the multi-protocol industrial data to the cloud in the heterogeneous network is solved; and the securityof data transmission is ensured comprehensively in the network, transmission and application layers, and a complete security guarantee system is formed.

Embodiments facilitate confidential and secure sharing of anonymous user profile data to improve the delivery of customized content. Embodiments of the invention provide a data appliance to an entity such as a business to convert profile data about the business's customers into anonymous identifiers. A similar data appliance is provided to a content provider in one embodiment to generate identifiers for its user profile data. Because the anonymous identifiers are generated with the same anonymization method, identical identifiers are likely generated from profile data of the same users. Therefore, the identifiers can be used to anonymously match the customers of the business to the users of the content provider. Therefore, data can be shared to improve customized content such as advertisements that the business wishes to place with the content provider without requiring the business to disclose customer data in an unencrypted form, and any non-matched data can remain confidential.

Methods, systems, and computer program products for automatically transforming data or business messages to enable communication between parties in a distributed computing environment (e.g. between business partners who exchange data over a public network, and who do not necessarily use the same data formats), where this data transformation preferably occurs at the edge of the computing network. A set of transformation services is described, and include template registration, transformation, forward, and receive services. Services from this set may optionally be combined to create composite transformation services. A template defines how to transform one data format into another. Selection criteria may be provided to enable dynamically selecting a template at run-time. Using these services, one business partner simply hands its data off into the network to a network-accessible transformation service, where the data is automatically transformed into a format usable by its business partner and forwarded to that business partner.