Is there a way I send mail with all files (as Zip) received in Spring Integration FTP instead sending the files Individually per Message received

Question

I have a Spring Boot application based on Spring Integration FTP where I can DOWNLOAD/UPLOAD files.

I need to send the files (as ZIP) received via mail but the problem is, I can only do this per message a file received which means if I receive 100 files, the email recipient would receive 100 emails. This is because Spring Integration processes the messages one after the other.

I need a way to aggregate the files received per poll and send to the recipient instead of sending the files one after the other.

I have tried to come up with some ideas such as these:

move the files received to a temporary location as they are received, check when the poll is done, zip the files in this temporary location, and send them to the recipient.
Find out how to know files received per poll.

Please can someone advise on how I can implement this:

public IntegrationFlow createIntFlow(MessageDirectory directory, ConcurrentMetadataStore metadataStore) {
        var directoryName = directory.getDirectoryName();

        var integrationFlowBuilder = IntegrationFlows
                .from(getInboundAdapter(directory, metadataStore),
                        e -> e.id(directoryName + "-PerPoller")
                                .autoStartup(true)
                                .poller(Pollers
                                        .cron(directory.getSchedule())
                                        .taskExecutor(simpleAsyncTaskExecutor)
                                        .errorChannel("errorChannel")
                                        .maxMessagesPerPoll(-1)))
                .log()
         

        **integrationFlowBuilder.publishSubscribeChannel(s -> {
           s.subscribe(sf -> sf.handle(c -> emailEmitter.createFileEvent(directory, c.getPayload().toString())));**
       });

            });
        }
        return integrationFlowBuilder.get();
    }```

I think an aggregator is the right component. https://docs.spring.io/spring-integration/reference/html/aggregator.html — René Winkler, Mar 07 '23 at 05:17
Thank you. I know Aggregator is an option. but please do you have an example of how this can be done ? — Chunks, Mar 07 '23 at 10:52

score 1 · Accepted Answer · answered Mar 07 '23 at 15:40

1

See this project https://github.com/spring-projects/spring-integration-extensions/tree/main/spring-integration-zip. There is a ZipTransformer which can zip an Iterable input message.

So, probably a source polling channel adapter does not fit to your requirements, but better to look into an FtpOutboundGateway and its MGET command support: https://docs.spring.io/spring-integration/docs/current/reference/html/ftp.html#using-the-mget-command. So, you got a List<File> as a result of this call and you send this message to that ZipTransformer. The result if zip is just one zip file which you simply can send to email.

You still may use a source polling channel adapter, but rather as a plain trigger via simple Supplier - () -> "". Or with a remote dir as a payload for that FtpOutboundGateway.

You may also consider to use a AcceptOnceFileListFilter as a transformer step before zipping to filter out those files you have processed before.

answered Mar 07 '23 at 15:40

Artem Bilan

113,505
11
91
118

Thank you Artem. I think the majority of my problem is not with zipping the files. it's how to aggregate the files when they are going through an integrationflow (without obstructing the flow) and send them via mail – Chunks Mar 09 '23 at 12:26
i used .aggregate(a -> {}...) which seems to work but i dont want it to stop the flow because of some other features using this implementation. I want to just aggregate the files and send to another service without stopping the initial flow – Chunks Mar 09 '23 at 12:30
I'm not sure what you mean with the stopping. If you think that an effect when aggregator emits nothing just because it is not ready to emit according to the `ReleaseStrategy` is a stopping, then it is wrong. An aggregator blocks nothing. If you don't want to use an `MGET`, then look into a `TimeoutCountSequenceSizeReleaseStrategy` or `group timeout`: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#agg-and-group-to – Artem Bilan Mar 09 '23 at 14:27

Is there a way I send mail with all files (as Zip) received in Spring Integration FTP instead sending the files Individually per Message received

1 Answers1