0

I use spring batch JpaItemWriter to insert db with 11500 records(8 columns for that table), and I thought its performance will be good, but it was not as fast as I expected. 11500 records took about 60 seconds with chunk size 100, then I raise the size to 10000, but it still took about 50 seconds. Could any one suggest better way to do bulk insert or update by spring batch?

a better way to insert or update large amount of data by spring batch

dandelion
  • 5
  • 4
  • Can you try inserting those records into a temp file and load it using native db command to load into the table? – gcpdev-guy Feb 23 '23 at 16:50

1 Answers1

0

The JpaItemWriter delegates item writing to a JPA EntityManager. So the performance of the writer is bound to the performance of the JPA implementation you use. What Spring Batch does is chunking items and then calling either entityManager.persist or entityManager.merge (depending on the usePersist parameter).

That said, raw JDBC inserts usually perform better than JPA, but this depends on the case. You need to try both approaches and compare the results.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50
  • 1
    Thanks for your answer, I tried to use jdbcBatchItemWriter, and the time used by this job reduced to about 3 seconds. I will test related update and delete operations again , really appreciate your answer – dandelion Feb 24 '23 at 03:14