A custom scoring method suitable for Lucene full-text search engine

Abstract

The invention discloses a self-defined scoring method suitable for Lucene full-text search engine, which is characterized by comprising: constructing a custom document collector, constructing a clause hit information collector, constructing a search clause assembly plug-in, constructing a Long-based type Numerical search and scoring plug-ins and other steps map the actual similarity ranking requirements to a limited number of Boolean clauses. According to the hit situation of different clauses and a search and scoring model based on Long-type numerical values ​​proposed by the present invention, the document's The final similarity score is mapped to the Long value in the form of high and low bits. Through the invention, not only a search and scoring plug-in with strong business scalability can be designed, but also the document similarity score output by the plug-in has high readability and interpretability, which brings convenience to development and debugging work. It reduces the difficulty of scoring and expanding the custom search based on the Lucne full-text search engine, and improves the efficiency of related work in the field.

Description

technical field [0001] The invention relates to the field of information retrieval, in particular to a self-defined scoring method suitable for a Lucene full-text retrieval engine. Background technique [0002] In the field of full-text retrieval, the open source project Apache Lucene, as the mainstream search engine in the industry, provides search service support for thousands of projects. Lucene not only excels in terms of performance, but also has good support in terms of scalability. Whether you use the Lucene engine directly or use the search engine system built on the Lucene engine, you can do personalized search customization. [0003] As the core part of full-text retrieval, the document similarity calculation process is often intervened and customized by different search services, and returns a customized score according to the actual scoring needs of the business. Although the default similarity algorithm provided by Lucene has a good performance in document sim...

AI Technical Summary

Problems solved by technology

[0005] 1) Arithmetic operation superimposed clause hit scoring method will encounter cross-class problems, which will lead to situations where the scoring value exceeds expectations

[0006] 2) When defining the scoring sub-sentence score, only a fixed score can be defined. When the weight of the scoring sub-sentence changes, it will bring a lot of adjustment costs

[0007] 3) Returning a meaningless similarity score to express the final similarity score is not intuitive enough

Although Lucene provides the Explanation method to output score explanations, for some custom scenarios, there is a certain maintenance cost

Images