0

Our production Domino server was 32-bit Windows 2003 server where OS, Program Files and Domino Data all were on different disk drives. I don't remember seeing this error on that server ever:

NotesException: Notes error: The full text index for this database is in use

After we switched to a 64-bit Windows 2008 virtual server with only C: drive we've seen this error a lot.

Today I made a test where I first generated 4000 rather large documents. Then I ran this XPage SSJS code:

for (var i = 0; i < 300; i++) {
    print("FT");
    database.FTSearch("[Form]=Test");
    java.lang.Thread.sleep(100);
}

While the above code was running I run this:

print("START indexing");
database.updateFTIndex(true);
print("END indexing");

I did this two times on both servers. On the 32bit dedicated server I got no errors. The indexing took 4 seconds and there were 32 FT searches while the indexing was ongoing.

On the 64bit virtual server both times there was only 1 FT search after the "START indexing" print and after that I got the error. All these 3 things occurres on the same second (start of indexing, one FT search and error).

Both Domino servers are version 9. On the virtual server there are more than 10GB available (FT index size in the test db is 10MB).

The only reason for this difference I can think of is that the FT indexer uses the other drives on dedicated server for indexing and because of that the error is not occurring there. Is that right and is there documentation about it?

We are not using FTBasePath notes.ini parameter.

Panu Haaramo
  • 343
  • 7
  • 20
  • You can use sysinternal tools to track what is touching the FT directory. In case it is anti-virus software/etc. http://technet.microsoft.com/en-us/sysinternals/bb896645 – Simon O'Doherty Aug 30 '13 at 12:13
  • That's a great idea, thanks! There is no anti-virus sw installed but I gave this a try. In the filter I included the path and file name without .nsf and excluded nhttp.exe and nserver.exe processed. I enabled only "Show file system activity". Then I reproduced the error but got no entries in the Process Monitor. – Panu Haaramo Aug 30 '13 at 19:07
  • Btw. don't leave Process Monitor running. I had filters on so almost no events were shown but they were all stored in the **page file**. I forgot the tool running and the server run out of disk space at night with a 20GB page file! – Panu Haaramo Aug 31 '13 at 15:10

2 Answers2

0

Having OS, Program Files and Domino Data on the same disk can/will be a performance issue, but should not cause the issue you are seeing.

As the production server apparently was migrated to the new platform,

  1. Remove all FT indexes (deleted the .ft directories) and recreate all FT indexes on the new machine. Stop the server, deleting the .ft directory of all relevant database, start the server again and recreate the FT index for those databases.
  2. Verify, that all the paths, ... in the notes.ini of the server are correct
  3. Update_Fulltext_Thread=1 is set?

Apart from that, verify that no virusscanners, ... are accessing the FT files, as Simon commented.

leyrer
  • 405
  • 2
  • 7
  • 1. No files have been copied at OS level. The FT indices have been created on the new server. I replicated my test db to the production server just before testing and created the FT index. 2. Yes they are. 3. It's set on both servers. – Panu Haaramo Aug 30 '13 at 19:15
0

I reduced the sleep time between FT searches to 10ms and added exception handling so that the FT search is tried again 10ms after the exception.

The result was 14 exception rounds on dedicated server and 12 on the virtual server. Probably there were some load on the virtual server when I first tested. 100ms sleep time was too long to cause the exception on the dedicated server.

So unfortunately this exception seems to occur always for a short moment when the full text indexing starts. On virtual server where other virtual servers are using the same RAID-system the disk operations can be sometimes slow causing the FT index busy time to increase.

Panu Haaramo
  • 343
  • 7
  • 20
  • If the issue is really the io, you should think about splitting things up. Take a look at "Building a high performance Domino Server" http://www.wissel.net/blog/d6plinks/SHWL-7RB3P5 – leyrer Sep 01 '13 at 20:21
  • Yes but it's quite limited what we can do in a virtual/cloud server by service provider. At some point we'll move to SSD storage which probably helps a lot (btw. SSD not mentioned in Stephan's old article). – Panu Haaramo Sep 02 '13 at 06:02
  • He mentions them, but yeah, they would probably help. :D – leyrer Sep 02 '13 at 13:41
  • Sorry you are right. I just searched the article with "SSD". – Panu Haaramo Sep 02 '13 at 14:03