1

I am trying to use Spring Integration with Spring batch.I am transforming my message Message to JobLaunchRequest

public class FileMessageToJobRequest {
private Job job;
private String fileParameterName;

public void setFileParameterName(String fileParameterName) {
    this.fileParameterName = fileParameterName;
}

public void setJob(Job job) {
    this.job = job;
}

public JobLaunchRequest toRequest(Message<File> message) throws IOException {
    JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
    String fileName = message.getPayload().getName();       
    String currentDateTime = LocalDateTime.now().toString();
    logger.info("currentDateTime"+currentDateTime);
    jobParametersBuilder.addString(this.fileName, fileName);
    jobParametersBuilder.addString(this.currentDateTime, currentDateTime);
    return new JobLaunchRequest(job,   jobParametersBuilder.toJobParameters());
}
}

here is my Integration flow

return IntegrationFlows.
                    from(Files.inboundAdapter(new File("landing"))
                                    .preventDuplicates()                                        .patternFilter("*.txt")
                                    , e -> e.poller(Pollers.fixedDelay(2500)                                                        .maxMessagesPerPoll(15)
                                        .taskExecutor(getFileProcessExecutor())))
                    .transform(fileMessageToJobRequest)
                    .handle(jobLaunchingGw(jobLauncher)).get();


@Bean
public MessageHandler jobLaunchingGw(JobLauncher jobLauncher)
{
    return new JobLaunchingGateway(jobLauncher);
}

For some reason one of the files of the two that print the same time is not getting processed if two files print the same currentDateTime.The files are most likely to come at the same time.I am using ThreadPoolTaskExecutor corepool size,max pool size ,queue capacity is 15

String currentDateTime = LocalDateTime.now().toString();

the file is not getting processed.I am assuming this is some kind of race condition.I even tried synchronized on the JobLaunchRequest.Is this a race condition?Why would a message drop?

I am using Spring Integration 4.2.6 all latest versions(almost) of Spring.

My reference for this config is this question Java DSL config

Community
  • 1
  • 1
Harish
  • 3,343
  • 15
  • 54
  • 75
  • OK. And how does it work if you don't use `.taskExecutor()`? Also, please, share `DEBUG` logs for `org.springframework.integration` and `org.springframework.batch` and point out where you should see the second file, but there is no such. – Artem Bilan Jul 21 '16 at 22:54
  • I wonder what you will have if you something like `.channel(new NullChannel())` after `.handle(jobLaunchingGw(jobLauncher))`. Just because `JobLaunchingGateway` is `AbstractReplyProducingMessageHandler`, so without an `outputChannel` you may end up with dead Threads in your pool. – Artem Bilan Jul 21 '16 at 22:57
  • The issue is intermittent but happens frequently.I have .handle(Files.outboundAdapter("headers['destinationPath']") .deleteSourceFiles(true)) .get(); after my jobLaunchingGateway.I am trying to recreate the issue with the Spring batch and Integration logs.Will post once I get them – Harish Jul 22 '16 at 04:23
  • I have identified the issue.I was using H2 Embedded Database for JobRepository.So when 2 files came in at the same time the DB transaction failed due to ConcurrentModificationException.Do you have any suggestion for non DB based thread safe JobRepository? – Harish Jul 27 '16 at 15:32

0 Answers0