1

in my camel app, it is process file from ftp server. When I test, I found during file upload, meantime my route start pick up that file and do process. I have set readLock to 'changed' and delay is '60000', my file is around 500m. Does I missing anything?

user6807013
  • 39
  • 1
  • 4
  • Beware that changed read lock for FTP servers often only have hh:mm for time (not seconds) so it can often not detect a change so fast. For FTP transfer its always better if the file is written using a temporary name and then renamed when done, or use extra marker files (done files) to signal when its ready to download. – Claus Ibsen Feb 11 '20 at 08:50
  • from camel doc, i found `markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component`, does this means marker files can't use in my situation - ftp component. – user6807013 Feb 12 '20 at 19:08
  • how to achieve `if the file is written using a temporary name and then renamed when done`? – user6807013 Feb 12 '20 at 19:19
  • Its the other system that uploads/writes those files. You are talking about downloading files. Are you using Camel for both upload and download? And if so then read the docs as its documented there about temporary files. – Claus Ibsen Feb 13 '20 at 04:33
  • Yes, there has another system to upload files. I only use camel to download files. So I guess set **readLockCheckInterval** to 1mins can help me with my problem. – user6807013 Feb 17 '20 at 01:10

1 Answers1

2

Notice that the delay option is just a fixed interval that is not "coupled" with the readLock.

The readLock option changed checks every second if the filesize has changed. With slow uploads this could be the reason that files still uploading are already be consumed.

You could try to increase the readLockCheckInterval higher than 1 second.

See Camel FTP docs for more details and options (Option readLock)

burki
  • 6,741
  • 1
  • 15
  • 31
  • `This interval is used for sleeping between attempts to acquire the read lock.` so during every check, the check mechanism seems should not change. – user6807013 Feb 12 '20 at 19:28
  • Yes, if for example Claus comment with hh:mm applies to your case you would have to set `readLockCheckInterval` to at least 1 minute to recognize a file that is still uploaded. Camel would then check every minute if the filesize has changed and continue to do so until the filesize does not change between two checks. This obviously leads to a notable delay for file consumption – burki Feb 13 '20 at 08:10