I am trying to create do a simple job that consists of reading lines from a csv file, converting them to a data model class and then saving them. The problem is that the Tomcat server does not start and the web application does not continue to run, but rather the application closes after the batch process is successful.
What I am currently trying to keep the application running after the batch job was successful and continue running the web application.
I am not using any pre-configuration either by annotating the BatchConfiguration with @EnableBatchConfiguration
(which is deprecated) or by extending DefaultBatchConfiguration
And here is the code from my own configuration:
@Configuration
public class EplMatchesBatchConfig {
private final String[] PARSED_MATCH_INPUT_FIELD = new String[]{"link_match", "season", "date", "home_team", "away_team", "result_full", "result_ht", "home_clearances", "home_corners", "home_fouls_conceded", "home_offsides", "home_passes", "home_possession", "home_red_cards", "home_shots", "home_shots_on_target", "home_tackles", "home_touches", "home_yellow_cards", "away_clearances", "away_corners", "away_fouls_conceded", "away_offsides", "away_passes", "away_possession", "away_red_cards", "away_shots", "away_shots_on_target", "away_tackles", "away_touches", "away_yellow_cards", "goal_home_ft", "goal_away_ft", "sg_match_ft", "goal_home_ht", "goal_away_ht", "sg_match_ht",};
@Autowired
private JpaTransactionManager transactionManager;
@Autowired
private DataSource dataSource;
@Bean
public JobRepository jobRepository2() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(dataSource);
factory.setTransactionManager(transactionManager);
factory.setIncrementerFactory(new DefaultDataFieldMaxValueIncrementerFactory(dataSource));
factory.setIsolationLevelForCreate("ISOLATION_READ_COMMITTED");
factory.setTablePrefix("BATCH_");
factory.afterPropertiesSet();
return factory.getObject();
}
@Bean
public FlatFileItemReader<ParsedMatchInput> reader() {
var tokenizer = new DelimitedLineTokenizer();
tokenizer.setDelimiter(";");
tokenizer.setNames(PARSED_MATCH_INPUT_FIELD);
var lineMapper = new DefaultLineMapper<ParsedMatchInput>();
lineMapper.setLineTokenizer(tokenizer);
// https://stackoverflow.com/questions/66234905/reading-csv-data-in-spring-batch-creating-a-custom-linemapper
lineMapper.setFieldSetMapper(new BeanWrapperFieldSetMapper<>() {{
setTargetType(ParsedMatchInput.class);
}});
return new FlatFileItemReaderBuilder<ParsedMatchInput>()
.name("matches")
// https://stackoverflow.com/questions/46403159/spring-batch-input-resource-must-exist-reader-is-in-strict-mode-error
.resource(new PathResource("./src/main/resources/assets/dataset/eplmatches2.txt"))
.lineMapper(lineMapper)
.linesToSkip(1)
.build();
}
@Bean
public ItemProcessor<ParsedMatchInput, Match> processor() {
return new MatchDataProcessor();
}
@Bean
public Job eplMatchesBatchJob(Step step1) throws Exception {
return new JobBuilder("EPL Parser", jobRepository2())
.incrementer(new RunIdIncrementer())
.flow(step1)
.end()
.build();
}
@Bean
public Step step1(JpaItemWriter<Match> writer) throws Exception {
return new StepBuilder("First and Final Step", jobRepository2())
.<ParsedMatchInput, Match>chunk(10, transactionManager)
.reader(reader())
.processor(processor())
.writer(writer)
.build();
}
}
Here is the writer that I have customized
@Component
@Slf4j
public class MatchWriter extends JpaItemWriter<Match> {
private final TeamRepository teamRepository;
private final MatchRepository matchRepository;
private final StatsRepository statsRepository;
MatchWriter(LocalContainerEntityManagerFactoryBean entityManagerFactory, MatchRepository matchRepository, StatsRepository statsRepository, TeamRepository teamRepository) {
super();
super.setEntityManagerFactory(Objects.requireNonNull(entityManagerFactory.getObject()));
this.matchRepository = matchRepository;
this.statsRepository = statsRepository;
this.teamRepository = teamRepository;
}
@Override
public void write(Chunk<? extends Match> items) {
super.write(items);
}
@Override
protected void doWrite(EntityManager entityManager, Chunk<? extends Match> items) {
items.forEach(match -> {
var matchKey = match.getMatchKey();
// log.info(matchKey.toString());
var homeTeam = match.getMatchKey().getHomeTeam();
var awayTeam = match.getMatchKey().getAwayTeam();
this.teamRepository.save(homeTeam);
this.teamRepository.save(awayTeam);
this.teamRepository.flush();
var homeTeamStats = match.getHomeTeamStats();
homeTeamStats.setMatchKey(matchKey);
this.statsRepository.save(match.getHomeTeamStats());
var awayTeamStats = match.getAwayTeamStats();
awayTeamStats.setMatchKey(matchKey);
this.statsRepository.save(match.getAwayTeamStats());
this.matchRepository.save(match);
});
}
}
This is the output that I am getting from the console
2023-03-20T18:05:51.685+01:00 INFO 51001 --- [ restartedMain] i.h.fergietime.FergieTimeApplication : No active profile set, falling back to 1 default profile: "default"
2023-03-20T18:05:51.703+01:00 INFO 51001 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2023-03-20T18:05:52.024+01:00 INFO 51001 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-03-20T18:05:52.056+01:00 INFO 51001 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 28 ms. Found 3 JPA repository interfaces.
2023-03-20T18:05:52.236+01:00 INFO 51001 --- [ restartedMain] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-03-20T18:05:52.258+01:00 INFO 51001 --- [ restartedMain] org.hibernate.Version : HHH000412: Hibernate ORM core version 6.1.7.Final
2023-03-20T18:05:52.453+01:00 INFO 51001 --- [ restartedMain] SQL dialect : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQLDialect
2023-03-20T18:05:52.796+01:00 INFO 51001 --- [ restartedMain] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-03-20T18:05:52.800+01:00 INFO 51001 --- [ restartedMain] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-03-20T18:05:52.934+01:00 INFO 51001 --- [ restartedMain] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: POSTGRES
2023-03-20T18:05:53.171+01:00 INFO 51001 --- [ restartedMain] efaultSchemaResourceGraphQlSourceBuilder : Loaded 1 resource(s) in the GraphQL schema.
2023-03-20T18:05:53.285+01:00 INFO 51001 --- [ restartedMain] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729
2023-03-20T18:05:53.307+01:00 INFO 51001 --- [ restartedMain] i.h.fergietime.FergieTimeApplication : Started FergieTimeApplication in 1.779 seconds (process running for 2.248)
2023-03-20T18:05:53.307+01:00 INFO 51001 --- [ restartedMain] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2023-03-20T18:05:53.374+01:00 INFO 51001 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=EPL Parser]] launched with the following parameters: [{'run.id':'{value=36, type=class java.lang.Long, identifying=true}'}]
2023-03-20T18:05:53.418+01:00 INFO 51001 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [First and Final Step]
2023-03-20T18:06:06.303+01:00 INFO 51001 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [First and Final Step] executed in 12s883ms
2023-03-20T18:06:06.317+01:00 INFO 51001 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=EPL Parser]] completed with the following parameters: [{'run.id':'{value=36, type=class java.lang.Long, identifying=true}'}] and the following status: [COMPLETED] in 12s930ms
2023-03-20T18:06:06.331+01:00 INFO 51001 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
Process finished with exit code 0