Spring Batch: Multiple Item Readers in a single step

Question

I am a newbie in spring batch. The task I need to achieve in spring batch as follows:

1. Need to read some metadata from database.
2. Based on this metadata, I need to read some files.
3. After some processing, need to write those values from file to database.

My queries are the following:

a. For the 1st requirement, I needed to map the whole resultset to a single object, where Person related data are in 1 table and Pets related data are in another table and joined by person id.

public class PersonPetDetails {

    private String personName;
    private String personAddr;

    private int personAge;

    private List<Pet> pets;

For this I have written a custom Item reader which extends JdbcCursorItemReader.

public class CustomJDBCCusrorItemReader<T> extends JdbcCursorItemReader<T> {

    private ResultSetExtractor<T> resultSetExtractor;


    public void setResultSetExtractor(ResultSetExtractor<T> resultSetExtractor) {
        this.resultSetExtractor = resultSetExtractor;
    }

    @Override
    public void afterPropertiesSet() throws Exception {
        setVerifyCursorPosition(false);
        Assert.notNull(getDataSource(), "DataSource must be provided");
        Assert.notNull(getSql(), "The SQL query must be provided");
        Assert.notNull(resultSetExtractor, "ResultSetExtractor must be provided");
    }


    @Override
    protected T readCursor(ResultSet rs, int currentRow) throws SQLException {      
        return resultSetExtractor.extractData(rs);
    }
}

Is this the correct way to achieve my requirement? Or is there a much better way?

b. AFAIK, in spring batch there cannot be a step with just a reader, without a writer. Hence, I cannot call another set of reader in a different step of the Job. Then, how can I call multiple readers in a single step?

c. Also, based on some condition I may need to call a third set of Reader. How can I conditionally call a reader in a step?

Thanks for going through my post. I know it is long. Any help is much appreciated. Also, i guess an example code snippet would help me better understand the point. :)

Answer 1

I would recommend as below

High Level Design:

1. Partitioner It will deal with list of persons. Note: there is not Pet data pulled at this point of time.

2. Reader It will get a list of Pet which are belong to a Person. Note: Reader will return a list of Pet specific to a Person only.

3. Processor Base on a Pet-Person you will process base on your requirement.

4. Writer Based on your requirement to write to DB.

Low Level Code snippet:

1. Partitioner

   public class PetPersonPartitioner implements Partitioner {
   
     @Autowired
     private PersonDAO personDAO;
   
     @Override
     public Map<String, ExecutionContext> partition(int gridSize) {
   
       Map<String, ExecutionContext> queue = new HashMap<String, ExecutionContext>();
   
       List<Person> personList = this.personDAO.getAllPersons();
       for (Person person : personList) {
   
         ExecutionContext ec = new ExecutionContext();
         ec.put("person", person);
         ec.put("personId", person.getId());
   
         queue.put(person.getId(), ec);
       }
   
     return queue;
     }
   }
   

2. Reader

   <bean id="petByPersonIdRowMapper" class="yourpackage.PetByPersonIdRowMapper" />
   
   <bean id="petByPesonIdStatementSetter" scope="step"
         class="org.springframework.batch.core.resource.ListPreparedStatementSetter">
       <property name="parameters">
           <list>
               <value>#{stepExecutionContext['personId']}</value>
           </list>
       </property>
   </bean>
   

public class PetByPersonIdRowMapper implements RowMapper<PersonPetDetails> {
    @Override
    public BillingFeeConfigEntity mapRow(ResultSet rs, int rowNum) throws SQLException {
        PersonPetDetails record = new PersonPetDetails();

        record.setPersonId(rs.getLong("personId"));
      record.setPetId(rs.getLong("petid");
      ...
      ...
}

3. Processor You can continue working on each PersonPetDetails object.