Scalable analysis platform for semi-structured data

Abstract

A data transformation system includes a schema inference module and an export module. The schema inference module is configured to dynamically create a cumulative schema for objects retrieved from a first data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. Dynamically creating the cumulative schema includes, for each object of the retrieved objects, (i) inferring a schema from the object and (ii) selectively updating the cumulative schema to describe the object according to the inferred schema. The export module is configured to output the data of the retrieved objects to a data destination system according to the cumulative schema.

Description

[0001] Cross References to Related Applications [0002] This disclosure claims priority to US Patent Application No. 14 / 213,941, filed March 14, 2014, and also claims the benefit of US Provisional Application No. 61 / 800,432, filed March 15, 2013. The entire disclosures of the above-mentioned applications are incorporated herein by reference. technical field [0003] The present disclosure relates to a scalable interactive database platform, and more particularly, to a scalable interactive database platform for semi-structured data incorporating storage and computation. Background technique [0004] The background description provided herein is for the purpose of generally presenting the context of the disclosure. The work of the presently mentioned inventors (insofar as described in this Background section) and aspects of the background description which may not have been prior art qualification at the time of filing are neither expressly nor implicitly considered to be p...

AI Technical Summary

Problems solved by technology

Although Hadoop is flexible, using Hadoop requires specialized technical administrators and programmers with deep knowledge, which are often difficult to find

Also, Hadoop is too slow to be interactive

Images