lucene.net slow when searching on only 500k records

Question

I am using lucene.net for searching product bill details. totally in this 500k (document) records & 6 fields added to created documents.

I use lucene.net version 3.0.3.0.

public DataSet Search(string criterion, string term)
 {

 string indexPath = System.Configuration.ConfigurationManager.AppSettings["LuceneIndexPathBillSearch"];
            Lucene.Net.Store.Directory directory = FSDirectory.Open(new DirectoryInfo(indexPath));
            Analyzer analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);

            IndexReader indexReader = IndexReader.Open(directory, true);
            Searcher searcher = new IndexSearcher(indexReader);
            var queryParser = new QueryParser(Lucene.Net.Util.Version.LUCENE_30, criterion, analyzer);

            queryParser.AllowLeadingWildcard = true;

            var query = queryParser.Parse(term.ToLower() + "*");
             var sort = new Sort(new SortField(term, SortField.STRING, false));

            TopDocs resultDocs = searcher.Search(query, null, indexReader.MaxDoc, sort);
            var topDocs = resultDocs.ScoreDocs;

            DataSet Product= new DataSet();
            DataTable dt = new DataTable();
            dt.Columns.Add("BillId");
            dt.Columns.Add("BillNo");
            dt.Columns.Add("BillDate");
            dt.Columns.Add("BillUniqueNo");
            dt.Columns.Add("ProductName");
            dt.Columns.Add("ProductCode");
            dt.Columns.Add("Status");
            dt.Columns.Add("IsServiceOrder");
            DataRow row = null;
            foreach (var hit in topDocs)
            {
                var documentFromSerach = searcher.Doc(hit.Doc);
                row = dt.NewRow();
                row["BillId"] = documentFromSerach.Get("BillId");
                row["BillNo"] = documentFromSerach.Get("BillNo");
                row["BillDate"] = documentFromSerach.Get("BillDate");
                row["BillUniqueNo"] = documentFromSerach.Get("BillUniqueNo");
                row["ProductName"] = documentFromSerach.Get("ProductName");
                row["ProductCode"] = documentFromSerach.Get("ProductCode");
                row["status"] = documentFromSerach.Get("status");
                row["IsServiceOrder"] = documentFromSerach.Get("IsServiceOrder");
                dt.Rows.Add(row);
            }
            Product.Tables.Add(dt);

            return Product;
   }

What am I doing wrong here? why it's taking 10 to 15 seconds searching for text from created index?

score 0 · Answer 1 · answered Mar 21 '16 at 13:40

The problem is probably that you are creating an IndexReader for every query. Lucene is probably spending time caching term values (amongst other things) which is costing you the time.

This is the equivalent of stopping and starting a database server for each query.

If your index is "stable" then just hold onto the instance of the searcher.

If not and you do updates that you want the search to see. Then keep and instance of the IndexWriter. Do all updates via that (it's threadsafe). Then create your searcher with searcher = new IndexSearcher(writer.GetReader()). This will ensure that the reader/searcher are always up to date.

Bottom line... don't create/recreate many instances of IW, IR, IS in your app. Keep a single instance and do all work through that.

There are many more nuances to this for perf/latency etc.

lucene.net slow when searching on only 500k records

1 Answers1