12

The application I am writing needs to be able to copy files that are locked. We attempted to use Volume Shadow Copy, and while it was successful in copying the file, the application that had the lock on the file crashed because it could not acquire a lock while we were copying the file.

I am left to believe that my only option is to bypass the OS and read directly from the disk. The problem is that if I read directly to the disk I cannot be sure of the integrity of the file, if it is in the middle of a write the file will be in a damaged state.

After hours of searching I was able to find one utility that copied the file directly from the disk and used a file system driver to cache writes while copying so that it could make sure that the file was in an intact state. However, that utility is extraordinarily expensive, 100k+ for the license I would likely need to use.

Does anyone have any ideas on how to accomplish what I am trying to?

We are planning on restricting the system to NTFS volumes only.

Eric Milas
  • 1,807
  • 2
  • 18
  • 29
  • Does it really need to live on a NTFS? I would try running the app under wine on linux on an ext4 or similar and just copy the file. – Gunther Piez Jun 06 '11 at 21:59
  • Yes, we do need to be on NTFS, we don't have control over the hardware. – Eric Milas Jun 06 '11 at 23:10
  • 3
    I'm a little bit confused why the application crashed when you were using shadow copy. This shouldn't happen. A shadow copy is a read-only copy of the whole filesystem. The locking mechanism should not be affected. – UrOni Jun 09 '11 at 11:05
  • From my understanding, VSS temporarily freezes the file system while it makes it's copy. For VSS to work properly the application would have to support VSS hooks. For example, SQL Server listens for VSS requests and when it receives one it will prepare itself for a copy and then will let VSS know that it is safe to copy the files. – Eric Milas Jun 09 '11 at 12:35
  • 1
    From my understanding the VSS hooks are only there that the application can bring the data into a consistent state if it is not designed in a way that it is always in a (crash) consistent state. – UrOni Jun 09 '11 at 14:46
  • 2
    @UrOni is correct. What you are attempting to do is the entire reason Volume Shadow Copy exists. If it is not working, either you mis-interpreted the error or there is a bug in Volume Shadow Copy. – Nemo Jun 14 '11 at 15:41

3 Answers3

1

Can you grab the process ID of the application that has a lock on it and suspend its thread while you perform the copy? Something like this http://www.codeproject.com/KB/threads/pausep.aspx

halfer
  • 19,824
  • 17
  • 99
  • 186
Geoffrey
  • 10,843
  • 3
  • 33
  • 46
  • Unfortunately no, the application needs to continue running and could very well be reading and writing from the file while we are copying it. – Eric Milas Jun 09 '11 at 02:36
  • What exactly is the application? or better yet, why do you need to take a backup like this? – Geoffrey Jun 09 '11 at 02:41
  • You could inject a dll into the target application that hooks the createfile/read/write/close functions and implement the caching you require, and then apply the changes after the file backup has been completed. – Geoffrey Jun 09 '11 at 02:44
  • How are you supposed to figure out when the application has the file in a consistent state? – nobody Jun 09 '11 at 02:47
  • That was not the question asked, Eric is asking for an alternative to the $100k+ software, no need for the downvote as my answer is correct, but does not suit his requirements. – Geoffrey Jun 09 '11 at 02:59
  • The application will vary. I need to read them for data mining purposes and the applications can not have any downtime. – Eric Milas Jun 09 '11 at 03:29
  • @gnif, yes that does sounds like what we need, I am hoping to find a library or utility to accomplish that because it would likely be very time consuming. – Eric Milas Jun 09 '11 at 03:33
  • @Andrew, that is a good question and something we are trying to figure out. The application i mentioned claims to do it. – Eric Milas Jun 09 '11 at 03:33
  • @Eric Milas, have a look at http://www.madshi.net/madCodeHookDescription.htm for a good hooking library. Your concept has intrequed me, I may have a go at developing something myself as I am sure that there are many cases where this would be useful. – Geoffrey Jun 09 '11 at 11:08
1

I ended up using a C program called DirectCopy written by Napalm. It works rather well.

http://www.rohitab.com/discuss/topic/24252-ntfs-directcopy-method-from-napalm/

Eric Milas
  • 1,807
  • 2
  • 18
  • 29
  • 1
    It should be noted that it does not verify the integrity of the file bile, but it will make direct copies of files without creating file locks. It was able to copy a file that was in use by an app where VSS copies were crashing the app. – Eric Milas Aug 03 '11 at 14:47
0

This description of "layered drivers" might be useful. I know nothing about it though.

Also, if the file is locked then can you just 'watch' it and wait for it to be unlocked and then quickly copy it?

Andy Johnson
  • 7,938
  • 4
  • 33
  • 50