Spring boot integration with spring batch and jpa

Question

I am integrating a spring boot project with a spring batch and data jpa project . All stuff related to job and data configuration is right except , persisting my job writer result in database . after I read a file and process it , i can't write it to mysql database . There is no error but no inserting too . interesting thing is my datasource is configured . because before inserting , I can fetch a sample record from database .please assist me to solve this problem.

my application.properties :

spring.datasource.url = jdbc:mysql://localhost:3306/batchtest?  characterEncoding=UTF-8&amp;autoReconnect=true
spring.datasource.username = root
spring.datasource.password = root
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

batch configuration:

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {

@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;

@Bean
public ResourcelessTransactionManager transactionManager() {
    return new ResourcelessTransactionManager();
}

@Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
    MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
    mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
    return mapJobRepositoryFactoryBean.getObject();
}

@Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
    SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
    simpleJobLauncher.setJobRepository(jobRepository);
    return simpleJobLauncher;
}
@Bean
public FlatFileItemReader<Person> reader() {
    FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
    reader.setResource(new ClassPathResource("sample-data.csv"));
    reader.setLineMapper(new DefaultLineMapper<Person>() {{
        setLineTokenizer(new DelimitedLineTokenizer() {{
            setNames(new String[] { "firstName", "lastName" });
        }});
        setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
            setTargetType(Person.class);
        }});
    }});
    return reader;
}
@Bean
public PersonItemProcessor processor() {
    return new PersonItemProcessor();
}
@Bean
public ItemWriter<Person> writer() throws Exception {
    return new PersonWriter();
}
@Bean
public Job importUserJob() throws Exception{
    return jobBuilderFactory.get("importUserJob")
            .incrementer(new RunIdIncrementer())
            .flow(step1())
            .end()
            .build();
}
 @Bean
public Step step1() throws Exception{
    return stepBuilderFactory.get("step1")
            .<Person, Person> chunk(1)
            .reader(reader())
            .processor(processor())
            .writer(writer())
            .build();
}

Dao class :

public interface PersonDao extends CrudRepository<Person,Integer> {
}

writer class :

public class PersonWriter implements ItemWriter<Person> {
@Autowired
PersonDao personDao;

@Override
public void write(List<? extends Person> items) throws Exception {
    LOGGER.info("Received the information of {} students", items.size());
    for(Person person:items)
    {
        LOGGER.info(String.format("inserting for customre %s %s", person.getFirstName(), person.getLastName()));
        Person tempPerson = personDao.findOne(1);
        personDao.save(person) ;
        LOGGER.info(String.format("person id : %d",person.getId()));
    }

}

tempPerson is an object for testing the jpa data . it fetches a person object with id 1 from database but next line there is no inserting to database with no error. just executing of the line and continue the loop.

I know nothing about Spring batch, but using a ResourceLessTransactionManager and a MapJobRepositoryFactory is documented as "for in-memory persistence, and testing purposes" (http://docs.spring.io/spring-batch/trunk/reference/htmlsingle/#inMemoryRepository). Try using a JPA transaction manager, and a real JobRepository. — JB Nizet, Jul 10 '16 at 07:54
You're right, I should have read the post more carefully. My issue is therefore different. I am including it here: http://stackoverflow.com/questions/38287298 — balteo, Jul 10 '16 at 09:06

score 7 · Accepted Answer · edited May 30 '17 at 09:18

A solution to this problem may be closer than expected. Did you simply try to change the name of the transactionManager bean? With a different name it won't be used by default by Spring Data JPA.

I reproduced your problem and then I simply switched from this:

@Bean
public ResourcelessTransactionManager transactionManager() {
    return new ResourcelessTransactionManager();
}

to this:

@Bean
public ResourcelessTransactionManager resourcelessTransactionManager() {
    return new ResourcelessTransactionManager();
}

And in my opinion that solved the problem. Remember that 'transactionManager' is the default bean name for transactionManager in Spring Data JPA (at least as far as I understand, Spring Boot auto-configures it unless it finds a Bean with that name, and if it does, it uses the one found--and your database transactions are going through a Resourceless one).

You can also skip this:

@Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
    return new MapJobRepositoryFactoryBean(transactionManager).getObject();
}

and call the bean directly (just to be 'more sure' that proper Transaction Manager is used with Batch):

@Bean
public JobRepository jobRepository() throws Exception {
    return new MapJobRepositoryFactoryBean(resourcelessTransactionManager()).getObject();
}

Let me know when you test it, and I hope that it was the main problem :)

score 5 · Answer 2 · answered Jul 13 '16 at 16:54

I may have missed it, but I don't see where you specify what DB access method you're using (JPA, Hibernate, JDBC, etc.). I'm assuming JPA, but I think your ItemWriter needs to extend one of the DB-aware ItemWriters (RepositoryItemWriter,JpaItemWriter,JdbcBatchItemWriter, HibernateItemWriter). The base ItemWriter expects you to manage the transaction and all resources yourself. Try using RepositoryItemWriter (or whichever is appropriate) instead. You may have to provide an EntityManager and make sure write is called from within a Transaction (e.g. some @Transactional method).

Spring boot integration with spring batch and jpa

2 Answers2

Linked