Metadata indexing module and method based on cloud platform

Abstract

The invention discloses a metadata indexing model and a metadata indexing method based on a cloud platform. The model comprises a full volume indexing module, for mainly solving the problem of building full volume indexes for legacy data and migration data; an increment indexing module, for adding metadata which has passed audit into an index database in an incremental manner; an index management module, for meeting the basal management requirement of indexing. The method establishes a set of SaaS service for building indexes for the metabata. A tenant stores metadata into a metadata storage center through a migrating or registering process, the manner of building full volume index is adopted for the migration data, an increment index interface is called and built and the manner of incrementing is adopted for the metadata stored through the registering process, and thus data is added into the index database; building the index by a distributed search tool Elasticsearch. According to the metadata indexing model and the metadata indexing method based on the cloud platform, the speed for building index is obviously improved, real-time search can be achieved, and good expandability is provided.

Description

[0001] technical field [0002] The invention relates to a metadata indexing model and method based on a cloud platform, belonging to the technical field of information processing. Background technique [0003] Metadata is structural data that describes resource information. With the continuous improvement of informatization, there are more and more types of industry-related metadata, and the data volume of metadata is also increasing. The demand for fast indexing of metadata It is also becoming more and more urgent. By establishing a cloud platform to provide tenants with indexing SaaS services, multiple customers can share the same system, saving operation and maintenance costs, and also conducive to data centralization. [0004] Common search solutions currently include: using search engine APIs to build on-site searches, Alibaba Open Cloud Search, and using open source search engine tools such as Lucene and Solr to build indexes, etc. Using the API interface provided b...

AI Technical Summary

Problems solved by technology

Using the API interface provided by the search engine is easy to use, but it is not flexible enough and cannot be expanded well; Ali Open Cloud Search, which is a cloud search service for structured data, has the characteristics of easy expansion, customization, and real-time performance. However, it cannot meet the needs of various types of metadata configuration, and the uploaded metadata has not passed the review process, so the integrity and correctness of the data cannot be guaranteed; using Lucene to create indexes has been widely used in existing metadata indexing, but The Lucene search tool itself is not based on distribution and does not support the multi-tenancy mechanism. The index data isolation mechanism is complex and the index creation speed is relatively slow. Although the Solr search tool supports distributed and easy to expand, it also does not support multi-tenancy mechanism

Images