6

Over the past few weeks the company I work for has been having issues on their server wherein a file gets locked by something, and can't be unlocked. These files are almost always files that are being actively accessed over Windows file sharing. I've downloaded the popular Unlocker program, but that is unable to find any lock on the file. openfiles.exe (or System Tools > Shared Folders > Open Files) does not show the file as being open by anything. Yet I'm unable to open it in anything, including notepad - the system gives me a permission denied error even when accessing the file as the local system administrator. The problem always goes away when the server is rebooted.

To make matters worse, I've also been unable to reproduce this error at all. It's frequency is anywhere from daily to a week of working perfectly fine. When the problem does happen, I have very little time to investigate it, because nine times out of ten the problem makes our production systems completely nonfunctional, and management wants the server rebooted.

The problem is also not limited to a single application. The majority of the file sharing traffic is in our production management software, Globalshop, so the problem usually manifests itself somewhere in there, however the most recent occurrence was in a Microsoft Access database completely separate from anything Globalshop.

Has anyone seen anything like this? Any idea what could cause it? Let me know if there's any more information you need.

We're currently running Windows Server 2003 Standard Edition.

[edit] I've tried checking permissions, but I'm disallowed access to those as well on the offending file. I can't open, move, delete, rename or even copy the file.

[edit2] I've managed to isolate two offending files. As long as no other production-interfering problems crop up, I've been authorized by management to keep the server online for the rest of the day to test them.

[edit3] Brettski, in comments, suggested that I look at the file permissions. I've taken a screenshot of the properties window of one of the offending files. As you can see, there are no permissions available. http://i43.tinypic.com/24xgpe8.png (This is opened as the system administrator with full access rights.)

Sean Edwards
  • 63
  • 1
  • 2
  • 6
  • When you cannot access the file, have you checked the file permissions, have they changed in any way. I know you said it clears itself when rebooting, but thought it's worth a check. – Brettski Jun 03 '09 at 14:11
  • The permissions I assume are fine, but I'm disallowed access to those as well on the offending file, so I can't say. – Sean Edwards Jun 03 '09 at 14:14
  • When on the server as local/domain admin you cannot view the access rights to the file? Normally if you can see the file (file scan) you can see it's access rights. Another symptom. – Brettski Jun 03 '09 at 14:25
  • Hmm. I'll take a closer look at the file properties next time it happens. – Sean Edwards Jun 03 '09 at 14:26
  • 1
    Does System Tools > Shared Folders > Open Files show the files as being open by anyone? – Brettski Jun 03 '09 at 14:27
  • No. The file doesn't appear there, or in the output of openfiles.exe – Sean Edwards Jun 03 '09 at 14:28
  • Brettski: I've updated the initial question with a link to the properties window for one of the offending files. – Sean Edwards Jun 03 '09 at 17:43
  • Just to be sure, you are opening the file from the server console (or remote desktop to server)? – Brettski Jun 03 '09 at 20:40
  • I've tried both locally on the server, and over remote desktop, with the same results on both. – Sean Edwards Jun 04 '09 at 12:52

5 Answers5

3

Grab Handle.exe (http://technet.microsoft.com/en-us/sysinternals/bb896655.aspx) and, on the server computer, run a "handle -a" redirecting the output to a text file. Search that for the affected filename. That should at least tell you what process on the server computer has an open handle to the file. I'd be interested to know what process has a handle to it. (I'm wondering about anti-virus software as a possibility...)

My condolences for having to deal with GlobalShop, BTW.

Edit:

McAfee VirusScan 8, eh? Have a look at this: https://kc.mcafee.com/corporate/index?page=content&id=KB52156

It refers to Microsoft Office files, but it sounds suspiciously like it could affect any type of file. See also:

http://forums.mcafeehelp.com/showthread.php?t=223545

http://www.tek-tips.com/viewthread.cfm?qid=1453943&page=1

https://web.archive.org/web/1/http://techrepublic%2ecom%2ecom/5208-6230-0.html?forumID=101&threadID=207897&messageID=2632853

Evan Anderson
  • 141,881
  • 20
  • 196
  • 331
  • This is my favorite part - no matter what tool I use to view open files, the offending file is not there. The filename is RQT20TX099.TXT, which appears nowhere in the output of handle.exe or openfiles.exe, and is nowhere in System Tools > Shared Folders > Open Files. As far as I can tell, it's really actually not open. And I thank you for your condolences. :) – Sean Edwards Jun 03 '09 at 17:38
  • And, the server is running McAfee VirusScan Enterprise 8. – Sean Edwards Jun 03 '09 at 17:48
  • That seems really promising. The problem seems to be in VirusScan 8.0, which is the version we have. I'll try upgrading. (Unfortunately I have no way to assert that it fixed the problem, as I haven't figured out how to reliably reproduce it yet.) – Sean Edwards Jun 04 '09 at 14:10
  • Intermittent problems piss me off! *smile* Keep us appraised. If nothing else, if after the upgrade you can run for a multiple of the time in which the problem was happening previously you can probably be fairly confident that you've squashed the bug. – Evan Anderson Jun 04 '09 at 14:12
  • You may be interested to know that today we found out (the hard way) that our automatic backup software stopped making nightly backups at precisely the same day that this issue started. – Sean Edwards Jun 05 '09 at 15:13
2

Procmon might be your best solution here as it is able to log what processes are using what files. Even if the file isn't showing up in openfiles procmon will log who opened it. Antivirus software is the usual culprit in these cases.

Jim B
  • 24,081
  • 4
  • 36
  • 60
1

I have been having a similar problem on our file server (Windows Storage Server 2003 R2 x64). Only small temporary files (usually Access .ldb "lock" files, but also temp files used for various processes), no pattern on location or size or name or type. Files were locked with no access to permissions; unlocker programs (I was using EMCO UnLock IT, to which I will give a thumbs-up in passing) did not see any programs holding the files as locked. Rebooting always fixed the problem, but that was a major PITA with everyone having some files in use on that particular server.

Based on comments here, I turned off our AV software on that server. Without rebooting, the file that was untouchable a moment ago vanished (it was a temp file that should have been deleted anyway, so this was A Good Thing).

AV software is CA eTrust Antivirus v. 7.1. I'm guessing (with the aid of hindsight) that Old Software + x64 OS = Occasional Bad Things. :-)

My thanks to Sean and Evan for the discussion.

0

Unlocker seems to have potential. I haven't had a chance to try it yet though, so let me know how it works.

0

This is a very old topic, yes. But the reason and solution is detailed in this page about Opportunistic File Locking behavior in Windows. See the content under the heading "MS Windows Oplocks and Caching Controls" towards the end of the page.

Link: https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html

The opening paragraph is the clue to why this is relevant (almost exactly what the OP faced):

There is a known issue when running applications (like Norton Antivirus) on a Windows 2000/ XP workstation computer that can affect any application attempting to access shared database files across a network. This is a result of a default setting configured in the Windows 2000/XP operating system. When a workstation attempts to access shared data files located on another Windows 2000/XP computer, the Windows 2000/XP operating system will attempt to increase performance by locking the files and caching information locally. When this occurs, the application is unable to properly function, which results in an “Access Denied” error message being displayed during network operations.

Hope it helps.