Spring-Batch Multi-line record Item Writer with variable number of lines per record

Question

I have the below requirement but am not able to decide on the approach to take:

I need to write data to a fixed format out put file where each record spans over multiple lines as seen below:

000120992599999990000000000000009291100000000000000000000000010000
000000000000000000000006050052570009700000050000990494920000111100
      ABCDE:WXYZ                                              0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
000000000000000000000006050052730005700001100000090494920000221200
      ABCDE:WXYZ                                              0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
000000000000000000000006050113110009700000000000000494920000311100
      ABCDE:WXYZ                                              0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
000012099259999999000000000000000929110000000000000000000000001000

This is one record from above example:

000000000000000000000006050052570009700000050000990494920000111100
      ABCDE:WXYZ                                              0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200
      descriptiongoesheredescriptiongoesheredescriptiongoesher0200

The first and last line are header and footer respectively. First line of each record contains several details. 2nd line have some other details with spaces.

I have a long description field which I need to divide into 56 characters sections and append those with record's 3rd line onwards.

So in some of the records, this may be just one line while in some it could be three lines as well.

I need guidance about how to design my itemwriter in above scenario.

Nik

score 7 · Accepted Answer · answered Oct 20 '11 at 19:58

there is a multiline records writer example in the official spring-batch-samples, search for multiline.xml and MultiLineTradeItemWriter

its basically the usual delegate principle, you just need a proper domain object with supposable a list of those 1..n intermediate lines

    public class MultiLineTradeItemWriter implements ItemWriter<Trade>, ItemStream {

    private FlatFileItemWriter<String> delegate;

    public void write(List<? extends Trade> items) throws Exception {
                List<String> lines = new ArrayList<String>();
              for (Trade t : items) {
              lines.add("BEGIN");
              lines.add("INFO," + t.getIsin() + "," + t.getCustomer());
              lines.add("AMNT," + t.getQuantity() + "," + t.getPrice());
              lines.add("END");
            }
            this.delegate.write(lines);
     }
    }

Thanks Michael.. going through the example!! – Vicky Oct 21 '11 at 05:32 — Vicky, Oct 21 '11 at 05:32

score 0 · Answer 2 · answered Jan 17 '14 at 11:33

you can do it in CustomExtractor.

for Example:

     public Object[] extract(T item) {
        List<Object> values = new ArrayList<Object>();

        BeanWrapper bw = new BeanWrapperImpl(item);
        for (String propertyName : this.names) {
            values.add( bw.getPropertyValue(propertyName) + "\r\n");
        }
        return values.toArray();
    }

score 0 · Answer 3 · answered Jan 05 '15 at 22:53

I had a similar problem writing multiple rows to a database. Because the job step will create a List of your items, how can you return a List from the processer to the writer? That would create a List of Lists, and the doWrite method in the Writer is not set up to handle that scenario.

I am using the 1.2.0 spring-boot-starter-parent (which gives me spring-core 4.1.3) along with hibernate (4.3.7), In my case I have 'receivables', and for every 1 receivable that I read from a csv file, I then might need to update or insert many receivables into my table. I am also using HibernateItemWriter.

My solution was to extend the HibernateItemWriter. The sessionFactory field is private, so I am passing it via the constructor and also using the setter (to accomadate the existing code). So my original code (which works fine for a single 'receivable') originally looked like this:

    @Bean
    public ItemWriter<Receivable> recivableWriter() {
        HibernateItemWriter<Receivable> hibernateItemWriter = new HibernateItemWriter<Receivable>(){{ setSessionFactory(sessionFactory); }};
        return hibernateItemWriter;
    }

After extending the HibernateItemWriter, and modifying my processor to return a List, my code changed to:

    @Bean
    public ItemWriter<List<Receivable>> receivableWriter() {
        HibernateListItemWriter<List<Receivable>> hibernateItemWriter = new HibernateListItemWriter<List<Receivable>>(this.sessionFactory){{ setSessionFactory(sessionFactory); }};
        return hibernateItemWriter;
    }

And my extended class looks like this (I might clean it up but this is my initial pass. I also wish the sessionFactory and clearSession fields were not private)

package com.work;

import java.util.ArrayList;
import java.util.List;

import org.hibernate.SessionFactory;

public class HibernateListItemWriter<Receivable> extends org.springframework.batch.item.database.HibernateItemWriter<Receivable> {

    public HibernateListItemWriter(SessionFactory sessionFactory) {
        super();
        this.sessionFactory = sessionFactory;
    }

    private SessionFactory sessionFactory; 
    private boolean clearSession = true;

    @Override
    public void write(List items) {
        List<Receivable> unwrappedItems = new ArrayList<Receivable>();
        List<List<Receivable>> wrappedItems = (List<List<Receivable>>)items;
        for (List<Receivable> singleList: wrappedItems) {
            unwrappedItems.addAll(singleList);
        }
        doWrite(sessionFactory, unwrappedItems);
        sessionFactory.getCurrentSession().flush();
        if(clearSession) {
            sessionFactory.getCurrentSession().clear();
        }
    }

}

Big shout out to grepcode.com for making life easier.

Spring-Batch Multi-line record Item Writer with variable number of lines per record

3 Answers3

Linked