Directory lock error with Lucene.Net usage in an ASP.NET MVC site

Question

I'm building an ASP.NET MVC site where I want to use Lucene.Net for search. I've already built a SearchController and all of its methods, but I'm getting an error at runtime that occurs when the SearchController is first initialized.

In SearchController, here's how I'm creating an IndexWriter:

public static string IndexLocation = HostingEnvironment.MapPath("~/lucene");
public static Lucene.Net.Analysis.Standard.StandardAnalyzer analyzer = new Lucene.Net.Analysis.Standard.StandardAnalyzer();
public static IndexWriter writer = new IndexWriter(IndexLocation,analyzer);

The error occurs on the last line. Here's the message that I'm getting:

Lucene.Net.Store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@C:\Users\Username\Desktop\SiteSolution\Site\lucene\write.lock

Furthermore, here's the stack trace:

[LockObtainFailedException: Lock obtain timed out: SimpleFSLock@C:\Users\Username\Desktop\SiteSolution\Site\lucene\write.lock]
   Lucene.Net.Store.Lock.Obtain(Int64 lockWaitTimeout) in C:\Users\Username\Desktop\Lucene.Net_2_9_2\src\Lucene.Net\Store\Lock.cs:107
   Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean create, Boolean closeDir, IndexDeletionPolicy deletionPolicy, Boolean autoCommit, Int32 maxFieldLength, IndexingChain indexingChain, IndexCommit commit) in C:\Users\Username\Desktop\Lucene.Net_2_9_2\src\Lucene.Net\Index\IndexWriter.cs:1827
   Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean closeDir, IndexDeletionPolicy deletionPolicy, Boolean autoCommit, Int32 maxFieldLength, IndexingChain indexingChain, IndexCommit commit) in C:\Users\Username\Desktop\Lucene.Net_2_9_2\src\Lucene.Net\Index\IndexWriter.cs:1801
   Lucene.Net.Index.IndexWriter..ctor(String path, Analyzer a) in C:\Users\Username\Desktop\Lucene.Net_2_9_2\src\Lucene.Net\Index\IndexWriter.cs:1350
   Site.Controllers.SearchController..cctor() in C:\Users\Username\Desktop\SiteSolution\Site\Controllers\SearchController.cs:95

[TypeInitializationException: The type initializer for 'Site.Controllers.SearchController' threw an exception.]

[TargetInvocationException: Exception has been thrown by the target of an invocation.]
   System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandle& ctor, Boolean& bNeedSecurityCheck) +0
   System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean fillCache) +86
   System.RuntimeType.CreateInstanceImpl(Boolean publicOnly, Boolean skipVisibilityChecks, Boolean fillCache) +230
   System.Activator.CreateInstance(Type type, Boolean nonPublic) +67
   System.Web.Mvc.DefaultControllerFactory.GetControllerInstance(RequestContext requestContext, Type controllerType) +80

[InvalidOperationException: An error occurred when trying to create a controller of type 'Site.Controllers.SearchController'. Make sure that the controller has a parameterless public constructor.]
   System.Web.Mvc.DefaultControllerFactory.GetControllerInstance(RequestContext requestContext, Type controllerType) +190
   System.Web.Mvc.DefaultControllerFactory.CreateController(RequestContext requestContext, String controllerName) +68
   System.Web.Mvc.MvcHandler.ProcessRequestInit(HttpContextBase httpContext, IController& controller, IControllerFactory& factory) +118
   System.Web.Mvc.MvcHandler.BeginProcessRequest(HttpContextBase httpContext, AsyncCallback callback, Object state) +46
   System.Web.Mvc.MvcHandler.BeginProcessRequest(HttpContext httpContext, AsyncCallback callback, Object state) +63
   System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData) +13
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +8682818
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +155

How can I resolve this issue?

UPDATE: I've started working on this particular project again, and it seems that I haven't fully resolved this issue yet.

The real issue is that the write.lock file isn't being removed after index usage ends. Based on the answer I have accepted, I understand the basic implementation logic, but I'm not sure if I have implemented it correctly. Here are some other methods in my class that are probably invalid:

    public ActionResult Search(string query)
    {

        var reader = writer.GetReader(); // Get reader from writer
        var searcher = new IndexSearcher(reader); // Build IndexSearch

        //Execute search...

        // Dispose of objects
        searcher = null;
        reader = null;

        return View();
    }

    public void AddToIndex(Document doc)
    {
        writer.AddDocument(doc);
        writer.Flush();
        writer.Optimize();
        writer.Flush();
    }

    private bool disposed = false;

    protected override void Dispose(bool disposing)
    {
        if (!disposed)
        {
            if (disposing)
            {
                // Release managed resources.
            }
            try
            {
                writer.Close();
                writer = null;
            }
            catch
            {

            }
            // Release unmanaged resources.
            // Set large fields to null.
            // Call Dispose on your base class.
            disposed = true;
        }
        base.Dispose(disposing);
    }

Any thoughts?

score 16 · Accepted Answer · answered Nov 22 '10 at 19:19

16

The reason why this happens is that the writer creates an empty file called write.lock as a cross-process lock. When that file exists, Lucene assumes that someone has a write lock on that directory.

When you terminate your process incorrectly, the file will not get deleted. So Lucene thinks that someone is still holding on to the lock. That's why you should always have a finally statement which closes the index.

If you are sure that the file is there in error (i.e. no Lucene processes are running) it is fine to just delete the file. Your index may, however, be in a corrupted state, since the writing was obviously terminated midstream.

answered Nov 22 '10 at 19:19

Xodarap

11,581
11
56
94

Thanks for your answer! My static IndexWriter never really gets closed... is that a problem? – Maxim Zaslavsky Nov 25 '10 at 06:49
@Maxim: The only reason that would be a problem is if multiple processes need to write. Otherwise, it's usually best to keep the writer open and let lucene decide when to flush to disk. – Xodarap Nov 25 '10 at 22:48
@Xodarap: I've started working again on this project, and I still haven't fully fixed the issue. I'm about to update the question with some of the code I'm using. I understand the basic implementation logic, but I'm not sure if I've implemented that correctly. – Maxim Zaslavsky Feb 19 '11 at 01:17
@Maxim: Are you getting errors on closing the index? On updating? – Xodarap Feb 19 '11 at 02:16
@Xodarap: the only issue is that the write.lock file that is being created isn't removed after the index usage ends. – Maxim Zaslavsky Feb 19 '11 at 04:30
@Maxim: and you're sure that the close() is being called without errors? Your sample code just had an empty catch, which worries me if there was some error you are getting. – Xodarap Feb 19 '11 at 16:43
@Xodarap: Nice notice! Let me go check what's going on there. – Maxim Zaslavsky Feb 19 '11 at 18:14
@Xodarap: huh, it seems like it's not being called at all. The way I'm modifying the index from another controller is as such: `var s = new SearchController(); s.ChangeIndex(post); s = null;`. When I do that, the **`Dispose` method isn't being called at all**. What did I implement incorrectly? Thanks! – Maxim Zaslavsky Feb 19 '11 at 19:54
1

@Xodarap: Never mind, fixed! I read the Lucene documentation, which says that closing the index should be done rarely. I changed all my closes to commits and only close when the application exits. It works now, thanks for your advice! :) – Maxim Zaslavsky Feb 20 '11 at 04:54

score 3 · Answer 2 · answered Nov 22 '10 at 05:48

3

Seems to be a dead-lock on lucene.

If supposedly NO index update into the collection,
simply remove this lock file C:\Users\Username\Desktop\SiteSolution\Site\lucene\write.lock.

After that re-run the index writing.

answered Nov 22 '10 at 05:48

ajreal

46,720
11
89
119

score 2 · Answer 3 · edited Sep 24 '12 at 06:36

2

try
{
    writer = new IndexWriter(directory, new StandardAnalyzer(), IndexWriter.MaxFieldLength.UNLIMITED);
}

catch (LockObtainFailedException ex)
{
     DirectoryInfo indexDirInfo = new DirectoryInfo(directory);
     FSDirectory indexFSDir = FSDirectory.Open(indexDirInfo, new Lucene.Net.Store.SimpleFSLockFactory(indexDirInfo));
     IndexWriter.Unlock(indexFSDir);
     writer = new IndexWriter(directory, new StandardAnalyzer(), IndexWriter.MaxFieldLength.UNLIMITED);
}

edited Sep 24 '12 at 06:36

Austin Henley

4,625
13
45
80

answered Sep 24 '12 at 06:05

Nayab

31
3

1

This code doesn't compile. DirectoryInfo constructor takes a string, not AzureDirectory object. Perhaps I am missing something. – Allen King Jun 29 '14 at 21:33
If you pass in the index location path string instead of the directory to DirectoryInfo(), this code works. – Tot Zam Jun 07 '16 at 18:07

score 0 · Answer 4 · answered Jan 07 '14 at 20:59

0

Researching this myself it seems that the indended approach would be to use multiple physical indexes and then merge them using the IndexWriter's .addIndexesNoOptimize or .addIndexes to merge all concurrent index changes.

Lucene documentation

answered Jan 07 '14 at 20:59

Niklas Arbin

652
6
14

score 0 · Answer 5 · answered Jun 29 '14 at 21:43

0

Haven't checked it myself but wouldn't this work (write after creating azureDirectory object)?

azureDirectory.ClearLock("write.lock")

answered Jun 29 '14 at 21:43

Allen King

2,372
4
34
52

score 0 · Answer 6 · answered Jul 04 '17 at 08:06

I encountered the same issue and I was using an IProviderContext. In my case I had to: Optimize,Commit and Dispose the ProviderContext.

providerContext.Optimize();
providerContext.Commit();
providerContext.Dispose();

I hope this helps. After implementing the above snippet, my index rebuilt successfully.

Directory lock error with Lucene.Net usage in an ASP.NET MVC site

6 Answers6

Linked