0

I have a scenario where there is an application pointing to a CIFS (Common Internet File System) and it read the files and then moves each over locally to process it and then deletes it, which all works fine.

However now the files needed to be read from the CIFS are getting more frequent and one box is no longer enough, so they have just rolled out more boxes with said application on which I presumed would be fine, but I am getting told that multiple machines are processing the same file, which I thought was odd, but I am not able to find out for sure how the CIFS standard works when dealing with multiple moves.

The steps in the scenario for each machine (which also has multiple threads with internal locking mechanisms):

  • Get list of all CIFS Files (matching a search crtieria)
  • Get a single file from the list
  • Check file has a corresponding file to indicate it has been written successfully
  • Move file to local disk
  • Delete corresponding files
  • Do stuff with file

Now my original assumption was that CIFS would only honour one move request at a given time, but from what is being reported to me I am unsure if this is the case and it is honouring both move requests.

Some more context to the environment/application:

  • CIFS hosted on Windows with username/password authentication
  • Application hosted on windows and written in .NET
  • Files moved using FileInfo.MoveTo()
  • All instances of applications are on different physical computers

Any info relating to the above would be great!

Grofit
  • 17,693
  • 24
  • 96
  • 176
  • I can't access the CIFS docs on this phone because they are in a self extracting zip (really, MS?) but at least one API for file rename/move in windows is not atomic so it would not surprise me if moving a file to a remote system was not atomic. Unlike local moves, this is really a move-then-delete operation. I recommend getting a exclusive lock (not an oplock) on the file before moving it. – John Watts Jun 19 '12 at 11:21
  • From experience, Id say multiple things can try move the file, just will fail at the end if either another machine has deleted it before it completes, or, it finishes and then cant delete because another has already done so. -- I would consider a "filname.lock" so if a file was grabbed, a .lock file indicates its done and is opened exclusively by the owner. – BugFinder Jun 19 '12 at 11:21
  • @BugFinder one of the current proposed solutions to this problem is using a custom *locking* mechanism, be it generating a new file or renaming the existing one. I just dont want to back these manual solutions (which themselves come with further problems i.e a file is locked but app dies, so its left in limbo) without making sure the work needs to be done. – Grofit Jun 19 '12 at 12:46
  • Appreciate that but there needs to be some form of official govener - even a cluster tends to have a lock disk to decide whos boss... – BugFinder Jun 19 '12 at 12:49
  • @JohnWatts I was hoping that in the case of it not automatically supporting atomic moves you could infer it via a lock request or something. That being said I have never worked with CIFS before this so dont know what I would need to do to request an exclusive lock, I will go google it, I also assume this would be a more low level P/Invoke based call not available at the higher .net File level? – Grofit Jun 19 '12 at 12:49

0 Answers0