3

I'm using Camel 2.15.2 with Apache Commons Net 3.3 on Java 8 deployed into a Tomcat container.

The issue is that consistently after processing just over 200 files (> 4000 files in the directory) the route stops, the FTP client disconnects and the following message is logged out: 

[ogs.sharp-stream.com:21/root/] FtpConsumer                    WARN  Error processing file RemoteFile[route/to/file] due to File operation failed:  Connection closed without indication.. Code: 421. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - File operation failed:  Connection closed without indication.. Code: 421] 
org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed:  Connection closed without indication.. Code: 421 
        at org.apache.camel.component.file.remote.FtpOperations.getCurrentDirectory(FtpOperations.java:713) 
        at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:440) 
        at org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:310) 
        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:384) 
        at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137) 
        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) 
        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) 
        at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) 
        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) 
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) 
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745) 
Caused by: org.apache.commons.net.ftp.FTPConnectionClosedException: Connection closed without indication. 
        at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:317) 
        at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:294) 
        at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:483) 
        at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608) 
        at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582) 
        at org.apache.commons.net.ftp.FTP.pwd(FTP.java:1454) 
        at org.apache.commons.net.ftp.FTPClient.printWorkingDirectory(FTPClient.java:2658) 
        at org.apache.camel.component.file.remote.FtpOperations.getCurrentDirectory(FtpOperations.java:709) 
        ... 15 more

This is the URI used at the begining of the related route.

As you can tell from the URI I'm also using a FileIdempotentRepository. It's defined like this 

            <property name="fileStore" value="target/fileidempotent/.filestore1.dat" />

            <property name="maxFileStoreSize" value="512000" />

            <property name="cacheSize" value="250" />
    </bean>

Any ideas why the connection might be closing before all files are processed?

Emzor
  • 1,380
  • 17
  • 28
Eric Ekong
  • 107
  • 1
  • 3
  • 8

2 Answers2

4

You need to add following parameter to you ftp/ftps url

disconnect=true

This will result in disconnecting from your remote host after file(s) are transferred and a fresh connection being attempted whenever the route is triggered again . (instead of using the stale one , which might have been closed by remote host/firewall)

AnkitPant
  • 56
  • 2
  • 2
    Just noting that a I got a similar issue with *disconnect=true* in my Camel URL, so it seems that this is a slip-through-case in which routes fail without any chance of handling errors? Unlike in other cases, such as invalid credentials, *CamelFailureEndpoint* isn't set and I don't get any indication of an error except the WARN log. – DoNuT Apr 04 '18 at 12:28
  • Maybe dumb question, but how does this look in an ftp url? In my case, trying to do a `hadoop fs -get ftp://usernamer:password@123.45.6.7/path/to/some/file?disconnect=true /some/local/path` and getting `No such file or directory` like it's trying to get a file named `path/to/some/file?disconnect=true`. – lampShadesDrifter Dec 26 '19 at 20:34
  • https://stackoverflow.com/questions/27930920/apache-camel-ftp-error suggests breaking into two routes, to force disconnect/reconnect – user77115 Jun 03 '20 at 10:39
0

I assume that your connection is closed from e.g. your fire wall. Try to set the parameter maxMessagesPerPoll to e.g. 150.

Shyam Bhimani
  • 1,310
  • 1
  • 22
  • 37
lahu89
  • 353
  • 1
  • 11