0

I am using FileSystemPersistentAcceptOnceFileListFilter and PropertiesPersistingMetadataStore to check if there is any new file (or) any file with same name (or) same file got modified then the payload should invoke the file. But it is not listening for the existing file got modified/modified with time.

Please suggest any good solution for resolving this issue

@Bean
@InboundChannelAdapter(value = "inputChannel", poller = @Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource<File> receive() throws Exception {
    FtpInboundFileSynchronizingMessageSource messageSource = new FtpInboundFileSynchronizingMessageSource(
            synchronizer());
    Logger.info(messageSource, "receive ");
    messageSource.setLocalDirectory(Temp);
    messageSource.setAutoCreateLocalDirectory(true);
    messageSource
            .setLocalFilter(fileSystemPersistentAcceptOnceFileListFilter());
    return messageSource;
}

@Bean
public FileListFilter<FTPFile> compositeFilter() throws Exception {

    Pattern pattern = Pattern.compile(".*\\.xml$");
    CompositeFileListFilter<FTPFile> compositeFileListFilter = new CompositeFileListFilter<FTPFile>();
    FileListFilter<FTPFile> fileListFilter = new FtpRegexPatternFileListFilter(
            pattern);
    compositeFileListFilter.addFilter(fileListFilter);
    compositeFileListFilter.addFilter(getAcceptOnceFileFilter());
    Logger.info(compositeFileListFilter.getClass().getName(), " compositeFilter ");
    return compositeFileListFilter;
}

@Bean
public FileListFilter<FTPFile> getAcceptOnceFileFilter() {

    FileListFilter<FTPFile> ftpPersistentAcceptOnceFileListFilter = null;
    try {
        ftpPersistentAcceptOnceFileListFilter = new FtpPersistentAcceptOnceFileListFilter(
                getMetadataStore(), "######");


    } catch (Exception e) {
        e.printStackTrace();
    }
    Logger.info(ftpPersistentAcceptOnceFileListFilter.getClass().getName(), " getAcceptOnceFileFilter ");
    return ftpPersistentAcceptOnceFileListFilter;

}

@Bean
public PropertiesPersistingMetadataStore getMetadataStore()
        throws Exception {
    PropertiesPersistingMetadataStore metadataStore = new PropertiesPersistingMetadataStore();
    metadataStore.setBaseDirectory("temp");
    metadataStore.afterPropertiesSet();
    Logger.info(metadataStore.getClass().getName(), " metadataStore ");
    return metadataStore;
}

@Bean
public AbstractInboundFileSynchronizer<FTPFile> synchronizer()
        throws Exception {
    AbstractInboundFileSynchronizer<FTPFile> fileSynchronizer = new FtpInboundFileSynchronizer(
            sessionFactory());
    fileSynchronizer.setRemoteDirectory("/RemoteFile/");
    fileSynchronizer.setDeleteRemoteFiles(false);
    fileSynchronizer.setFilter(compositeFilter());
    Logger.info(fileSynchronizer.getClass().getName(), " fileSynchronizer ");
    return fileSynchronizer;
}

@Bean
public FileSystemPersistentAcceptOnceFileListFilter fileSystemPersistentAcceptOnceFileListFilter() {
    ConcurrentMetadataStore metaDataStore;
    FileSystemPersistentAcceptOnceFileListFilter fileSystemPersistentFilter = null;
    try {
        metaDataStore = getMetadataStore();
        fileSystemPersistentFilter = new FileSystemPersistentAcceptOnceFileListFilter(
                metaDataStore, "######");
        fileSystemPersistentFilter.setFlushOnUpdate(true);

        return fileSystemPersistentFilter;
    } catch (Exception e) {
        e.printStackTrace();
    }
    Logger.info(fileSystemPersistentFilter.getClass().getName(), " fileSystemPersistentFilter ");
    return fileSystemPersistentFilter;

}

@Bean(name = "sessionFactory")
public SessionFactory<FTPFile> sessionFactory() throws SocketException, IOException {
    DefaultFtpSessionFactory ftp = new DefaultFtpSessionFactory();
    ftp.setHost(hostName);
    ftp.setUsername(username);
    ftp.setPassword(passWord);
    ftp.setBufferSize(1000);

    return ftp;
}

@Bean(name = "inputChannel")
public PollableChannel inputChannel() {
    QueueChannel channel = new QueueChannel();
    return channel;
}

@Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {

    PollerMetadata pollerMetadata = new PollerMetadata();
    pollerMetadata.setTrigger(new PeriodicTrigger(10));
    return pollerMetadata;
}

@ServiceActivator(inputChannel = "inputChannel")
public void foo(String payload) {

    System.out.println("payload: " + payload);
}

}

user3428736
  • 864
  • 2
  • 13
  • 33

1 Answers1

1

The FileSystemPersistentAcceptOnceFileListFilter is for local files, after the transfer already. To meet your requirements there is similar FtpPersistentAcceptOnceFileListFilter for remote entries.

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • Thanks Artem!! I have added the filter as mentioned in the code, but the filter is not looking for already existing files which are modified. Please have a look into the code and help me to fix this issue. – user3428736 Jun 22 '16 at 14:50
  • Does not make sense. Please, try to debug `AbstractInboundFileSynchronizer.synchronizeToLocalDirectory()` when it does `List filteredFiles = filterFiles(files);`. Don't do ` fileSynchronizer.synchronizeToLocalDirectory(Temp);` and ` fileSynchronizer.afterPropertiesSet();` manually. And make `synchronizer()` as a `@Bean` finally! – Artem Bilan Jun 22 '16 at 14:57
  • Thanks Artem!! I did the same changes as you have mentioned but I didn't able to copy the any files now. I am trying to transfer xml files where filter prefix can be "*.xml" and i am using the metastore for both the filter. Please let me know where i am going wrong. – user3428736 Jun 22 '16 at 15:40
  • You use the same `metadataStore` for both remote and local filters and both of them use the same `"*.xml"` `prefix`, which isn't a file name pattern as you may expect. Please, consider different prefixes, plus make `MetadataStore` as `@Bean` finally! I asked you to debug the code. Since you have so much mess I'm not sure how to help you yet. Try also to clean up the local dir and `metadata` properties files before the next test cycle. – Artem Bilan Jun 22 '16 at 15:48
  • Thank Artem!! The prefix, I have added prefix ".*\\.xml$" and modified metadatastore to @Bean and it started looking for new files.when I try to debug where my both filters are configured but if I place the existing file my debug point is not coming to ServiceActivator method. I doesn't get why if I modify the same and even it is synched with local folder it is not picking the modified file. – user3428736 Jun 22 '16 at 17:16
  • `Prefix` isn't suffix. Please, take a look to the `FtpPersistentAcceptOnceFileListFilter` ctor JavaDocs. For the pattern matching logic you still have to use `FtpRegexPatternFileListFilter` but together with the `FtpPersistentAcceptOnceFileListFilter` within the `CompositeFileListFilter`. So, both logic will work. You have to use different prefixes for both your filters. And I recommended you to debug exactly `AbstractInboundFileSynchronizer.synchronizeToLocalDirectory()` not your one. I can ask about DEBUG logs for `org.springframework.integration`, but I'm afraid we won't see there so much. – Artem Bilan Jun 22 '16 at 17:24
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/115395/discussion-between-user3428736-and-artem-bilan). – user3428736 Jun 23 '16 at 10:44
  • Thanks Artem!! I have fixed the issue using both the filters FileSystemPersistentAcceptOnceFileListFilter and FtpPersistentAcceptOnceFileListFilter and once copied, I am deleting the files in my local, so it will look for the same file again and it is working!!! – user3428736 Jun 27 '16 at 11:47
  • Comment posted in [answer](http://stackoverflow.com/a/39896943/5292302) by [viruskimera](http://stackoverflow.com/users/1734275/viruskimera), *I have a similar code but I get @Poller can not be defined as a type, any idea why? for more info see http://stackoverflow.com/questions/39875217/spring-ftp-inbound-channel-adapter* – Petter Friberg Oct 06 '16 at 13:15