0

I'm currently using Apache Derby to store data to a local database. Since I could be processing many thousands of records, I want to batch the commits for efficiency gains.

But if my batch size is 50, how do I commit to the database if I only have 20 records?

Many thanks!

jdie8274j
  • 155
  • 1
  • 10

1 Answers1

2

You have to commit a short batch of only 20 records in this case. You don't want to leave them hanging around in memory, because you can only trust stuff that has been written to the database to survive crashes and shutdowns.

If performance is a big deal, you will need to do experiments with tuning and database design. Based on experience with other databases, I suggest minimising the number of indexes to improve insert performance at the risk of making querying less efficient, and trying different database page sizes, including one that matches the block size of the underlying store, if you know it.

mcdowella
  • 19,301
  • 2
  • 19
  • 25