Lucene索引，查詢及高亮顯示

本文通過代碼簡單展示了采用TermQuery和FuzzyLikeThisQuery進行索引查詢，並且展示了如何在查詢結果中高亮顯示匹配的關鍵字（這在實際使用中是一個很有用的功能）

public class Indexer
{

    /**
     * @param args
     * @throws IOException
     * @throws LockObtainFailedException
     * @throws CorruptIndexException
     * @throws InvalidTokenOffsetsException 
     */
    public static void main(String[] args) throws CorruptIndexException,
            LockObtainFailedException, IOException, InvalidTokenOffsetsException
    {
        Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_36);

        IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_36, analyzer);
        config.setOpenMode(OpenMode.CREATE_OR_APPEND);
        
        Directory indexDir = new RAMDirectory();

        /**
         * 1. Indexing...
         */
        IndexWriter writer = new IndexWriter(indexDir, config);

        File docs = new File("D:\\files");

        if (docs.exists() && docs.isDirectory())
        {
            File[] files = docs.listFiles();
            
            if (files != null && files.length > 0)
            {
                for (File file : files)
                {
//                    •Field.Index.NO 不索引，如果存儲選項為YES，一般用於只保存不搜索的字段；
//                    •Field.Index.ANALYZED 分詞建索引；
//                    •Field.Index.NOT_ANALYZED 建索引但不分詞，字段雖然被索引但是沒有任何分析器對字段進行分析，只能整詞精確搜索，可保存唯一性字段（例如ID）並用於更新索引
                    Document doc = new Document();
                    doc.add(new Field("path", file.getAbsolutePath(), Field.Store.YES, Field.Index.NO));
                    doc.add(new Field("id", file.getName(), Field.Store.YES, Field.Index.NOT_ANALYZED));
                    doc.add(new Field("name", file.getName(), Field.Store.YES, Field.Index.ANALYZED, Field.TermVector.WITH_POSITIONS_OFFSETS));
                    
                    doc.add(new Field("size", file.getTotalSpace() + "b", Field.Store.YES, Field.Index.NO));
                    
                    writer.addDocument(doc);
                }
                
                writer.commit();
            }
        }
        
        writer.close(true);
        
        
        /**
         * 2. List indexed files ...
         */
        IndexReader reader = IndexReader.open(indexDir);
        IndexSearcher searcher = new IndexSearcher(reader);
        
        System.out.println("Max doc:" + searcher.maxDoc());
        System.out.println("List files below....");
        
        Document doc = null;
        for (int i = 0; i < searcher.maxDoc(); i++)
        {
            doc = searcher.doc(i);
            System.out.println("Doc " + i + " Name: " + doc.get("name") + ", Path: " + doc.get("path") + ", Size: " + doc.get("size"));
        }
        System.out.println("===================================================================================");
        
        
        /**
         * 3.Searching...
         */
        String id = "we";
        // 此處若改為Query queryId = new TermQuery(new Term("id", id));則無法搜索出結果，除非id = "We are young.txt";
        Query queryId = new TermQuery(new Term("name", id));
        TopDocs hitsForId = searcher.search(queryId, null, 100);
        if (hitsForId != null && hitsForId.totalHits > 0)
        {
            System.out.println("Searched " + hitsForId.totalHits + " docs for id " + id + "...");
            
            for (int j = 0; j < hitsForId.scoreDocs.length; j++)
            {
                System.out.println("Score doc for id " + j + " is " + hitsForId.scoreDocs[j].toString());
            }
        }
        System.out.println("===================================================================================");
        
        String keyword = "we are yy";
        FuzzyLikeThisQuery fuzzyLikeThisQuery = new FuzzyLikeThisQuery(100, analyzer);
        fuzzyLikeThisQuery.addTerms(keyword, "name", 0.8F, 0);
        
        // FuzzyLikeThisQuery不是lucene core自帶的查詢類，屬於contrib的query模塊
        // 默認情況下QueryScorer的私有成員WeightedSpanTermExtractor無法識別它，getBestFragment將返回null
        // 因此此處調用rewrite生成一個WeightedSpanTermExtractor可以識別的query對象，用於匹配內容關鍵字
        Query query = fuzzyLikeThisQuery.rewrite(reader);
        
        // 高亮顯示關鍵字，如果內容中本來就有<span></span>，可能導致顯示錯亂
        SimpleHTMLFormatter simpleHTMLFormatter = new SimpleHTMLFormatter("<span>", "</span>");
        Highlighter highlighter = new Highlighter(simpleHTMLFormatter, new QueryScorer(query));  
        
        TopDocs hits = searcher.search(fuzzyLikeThisQuery, null, 100);
        
        if (hits != null && hits.totalHits > 0)
        {
            System.out.println("Searched " + hits.totalHits + "docs for keyword " + keyword + "...");
            
            ScoreDoc[] sDocs = hits.scoreDocs;
            
            Document docMatched = null;
            for (int j = 0; j < sDocs.length; j++)
            {
                System.out.println("Score doc " + j + " is " + sDocs[j].toString());
                
                docMatched = searcher.doc(sDocs[j].doc);
                
                TokenStream tokenStream = analyzer.tokenStream("name", new StringReader(docMatched.get("name")));  
                String str = highlighter.getBestFragment(tokenStream,  docMatched.get("name"));
                
                System.out.println("Score doc " + j + " hightlight to: " + str);

            }
        }
        
        reader.close();
        indexDir.close();
    }
}

 

輸出如下

Max doc:13
List files below....
Doc 0 Name: ab.txt, Path: D:\files\ab.txt, Size: 104857595904b
Doc 1 Name: abc.txt, Path: D:\files\abc.txt, Size: 104857595904b
Doc 2 Name: M_1.txt, Path: D:\files\M_1.txt, Size: 104857595904b
Doc 3 Name: M_11.txt, Path: D:\files\M_11.txt, Size: 104857595904b
Doc 4 Name: We are young.txt, Path: D:\files\We are young.txt, Size: 104857595904b
Doc 5 Name: 什么是微博.txt, Path: D:\files\什么是微博.txt, Size: 104857595904b
Doc 6 Name: 喝水不忘挖井人.txt, Path: D:\files\喝水不忘挖井人.txt, Size: 104857595904b
Doc 7 Name: 天蒼蒼野茫茫.txt, Path: D:\files\天蒼蒼野茫茫.txt, Size: 104857595904b
Doc 8 Name: 怎么使用lucene.txt, Path: D:\files\怎么使用lucene.txt, Size: 104857595904b
Doc 9 Name: 神馬是一種馬嗎.txt, Path: D:\files\神馬是一種馬嗎.txt, Size: 104857595904b
Doc 10 Name: 蒼井.txt, Path: D:\files\蒼井.txt, Size: 104857595904b
Doc 11 Name: 蒼白 - 副本.txt, Path: D:\files\蒼白 - 副本.txt, Size: 104857595904b
Doc 12 Name: 蒼白.txt, Path: D:\files\蒼白.txt, Size: 104857595904b
===================================================================================
Searched 1 docs for id we...
Score doc for id 0 is doc=4 score=1.7948763 shardIndex=-1
===================================================================================
Searched 1docs for keyword we are yy...
Score doc 0 is doc=4 score=0.625 shardIndex=-1
Score doc 0 hightlight to: <span>We</span> are young.txt