3

i have a project with a spring integration flow defined as follows:

@Bean
public IntegrationFlow validationIntegrationFlow( ValidationService<String> validationService, ReleaseStrategy bundleReleaseStrategy ) {
    return IntegrationFlows.from( FtpIntegrationFlowConfiguration.BANK_FTP_CHANNEL )                
            .split()
            .enrichHeaders( headerEnricherSpec -> {
                headerEnricherSpec.headerFunction( DESTINATION_CHANNEL_HEADER, message ->
                        validationService.validate( ( String ) message.getPayload() ) ?
                                BANK_FTP_VALID_CHANNEL : BANK_FTP_BAD_CHANNEL );
            } )
            .aggregate( aggregatorSpec -> aggregatorSpec
                    .correlationStrategy( message -> message.getHeaders().get( DESTINATION_CHANNEL_HEADER ) )
                    .releaseStrategy( bundleReleaseStrategy )
                    .expireGroupsUponCompletion( true )
                    .groupTimeout( 1000 )
                    .sendPartialResultOnExpiry( true ) )
            .route( new HeaderValueRouter( DESTINATION_CHANNEL_HEADER ) )
            .get();
}

what this does is validate each message and assign the proper destination header then aggregates the results in groups and send that over to the router to dispatch to the correct channel

i have a service that checks the flow so that when it starts it logs how many messages will be sent and after the 2 destination channel i have 2 handles that tell the service that a bundle of X size was handled,

below there is a sample of the logs that i get

2016-12-09 17:04:00.176 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | synchronize | Start sync for correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49] with value [7956]
2016-12-09 17:04:01.397 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [6956]
2016-12-09 17:04:01.752 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [5956]
2016-12-09 17:04:02.114 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [4956]
2016-12-09 17:04:02.410 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [3956]
2016-12-09 17:04:02.681 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [2956]
2016-12-09 17:04:02.991 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [1956]
2016-12-09 17:04:03.344 | [taskScheduler-2] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1000] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [956]
2016-12-09 17:04:05.538 | [taskScheduler-5] | INFO | org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler | expireGroup | Expiring MessageGroup with correlationKey[bad.channel]
2016-12-09 17:04:05.538 | [taskScheduler-5] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [1] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [955]
2016-12-09 17:04:05.590 | [taskScheduler-9] | DEBUG | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | Subtract [955] from correlationId [b5e69b73-aa99-4e9e-a7d2-59c015793a49]. Result is [0]
2016-12-09 17:04:06.635 | [taskScheduler-2] | WARN | service.sync.impl.SynchronizationServiceInMemory | subtractAndGet | counter for [b5e69b73-aa99-4e9e-a7d2-59c015793a49] not found, ignoring subtract of [955]

as you can see after all messages have been processed i still log a warning after the ExpireGroup log that tells me another bundle was created.

i tried putting logs before the aggregator and no duplicate message appears there, can anyone help me configure the aggregator so that i don't get duplicates?

ps: if it can be of any help here is how i instantiate the bundle release strategy

new TimeoutCountSequenceSizeReleaseStrategy( 100, 1000L );
davide bubz
  • 1,321
  • 13
  • 31
  • You should explain why do you correlate by that header, but not default one which is exactly correlationId after splitting. That isn't clear also why did you decide that you have some duplication – Artem Bilan Dec 10 '16 at 00:47
  • I correlate by that header because in some cases I had the impression the default one was not propagated so I wanted to use a custom header. I know there is a duplicate because if you look at the logs all "bundles" are made of 1000 elements while the last 2 are 955 and you can see the service gets called twice with 955 also in the other bundles the group doesn't expire so it seems to me that this duplicates are caused by that expiration – davide bubz Dec 11 '16 at 12:57
  • That is your custom logs based on the correlation key header created by the splitter, so that doesn't prove duplications. Would be glad to have some test-case to play from my side. – Artem Bilan Dec 11 '16 at 14:24
  • 1
    I have been trying to reproduce with a test but for now i only see it when the whole application Is running but I will try to make a simple application that shows the issue. I'll update the question when I have that ready – davide bubz Dec 11 '16 at 14:31

0 Answers0