ItemWriter commit juste the first commit-interval

Question

I'm trying to select data from database, update each object and then update the database in an itemwriter.

I tried to flush DAO after each update but it's change nothing.

The configuration is very basic with a reader, a writer and a commit-interval of 100.

The reader is working as expected:

@Override
public Order read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
    if(iterator == null) {
        List<Order> all = findOrders();
        iterator = all.listIterator();
    }

    if (iterator.hasNext()) {
        return iterator.next();
    } else {
        return null;
    }
}

The writer is very basic too:

public void write(List<? extends Order> items) throws Exception {
    @SuppressWarnings("unchecked")
    List<Order> listOrder = (List<Order>) items;

    try {
        for(Order o : listOrder) {
            etatCommandeServiceImpl.updateEtatCommande(o);
        }
    }catch(Exception e) {
        if (log.isErrorEnabled()) {
            log.error("ERROR {}", e);
        }
        throw e;
    }
}

The issue is that the first 100 records are commited but not the rest. Spring batch table shows that it reads all records and commits multiple times but when I check in the database it's just commit once.

The version of Spring-batch is 2.2.6.

UPDATE

I think this problem is due to the transaction with database, because now it doesn't commit at all to the database. But I can't figure it out for the moment.

What are you doing inside etatCommandeServiceImpl.updateEtatCommande(o) method? — Tech Guy, May 06 '19 at 14:05
It just updates data. I don't think this point is problematic because it already works with a request hard-coded in the XML. But I need to go through classes to build request dynamically. — Youri C., May 06 '19 at 14:30
Are you sure your reader is not returning null after the first chunk? — Mahmoud Ben Hassine, May 07 '19 at 06:30
I checked, it returns null when the iterator has no more elements — Youri C., May 10 '19 at 14:27

score 1 · Answer 1 · answered May 06 '19 at 15:37

1

The issue is that you are returning null from your ItemReader. If you return null Spring batch would assume end of the feed is reached and it would terminate the job. Since you are returning null after every chunk (100 in your case),Sprint batch is getting terminated after processing 100 records.

Please create method with @PostConstruct and fetch all the records that you want to process and change your ItemReader to return 1 at a time and null when all the records are read.

answered May 06 '19 at 15:37

Tech Guy

417
2
7
23

But it already the case you've described. The first call of the reader retrieves data from database and then it calls iterator to return element one by one to the writer, it returns null when the iterator has no more elements, I just checked – Youri C. May 10 '19 at 14:26
Where exactly are you using chunk size? – Tech Guy May 10 '19 at 15:40
In my job.xml ``` ``` – Youri C. May 13 '19 at 07:06

ItemWriter commit juste the first commit-interval

1 Answers1