0

I am trying to collect huge data which may takes to complete around 5 days using JBeret implementation.We are running the extraction using Wildfly 10.1.0 Application Server with subsystem(jberet) as in-memory job-repository.

I took the chunk process for collecting data from database and batchlet for zipping process as two stepping process under single job id.

Also i am running the extraction using multi-threading which means we are collecting the data parallel using 10 Threads.

Due to Database slowness/memory issue faced the job failure exception.

2018-01-07 00:49:24,999 ERROR [org.jberet] (Batch Thread - 8) JBERET000007: Failed to run job simple-batchlet-job, step1, org.jberet.job.model.Chunk@3f63b3dd: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffffac1d2026:37db6cf6:5a4f7329:49355
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1190)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)

Is there is any possibility to pause all the Threads if there is any abnormal issues faced in database and resume it back so that we can flush out the garbage from database.

Thanks.

zen
  • 5
  • 5
  • There is no way to pause a job execution. You can stop a running job execution, which can be restarted later. The primary mechanism of error handling in batch processing is skip and retry depending on exception types. But in your case, the bottleneck is in db, which IMO should be the main focus of tuning. – cheng Jan 09 '18 at 14:53
  • You can explore various listeners (e.g., [ItemWriteListener](https://docs.oracle.com/javaee/7/api/javax/batch/api/chunk/listener/ItemWriteListener.html)) to see if they provide any hook for you to do any health check on the db, and probably perform some remedy there. – cheng Jan 09 '18 at 14:58
  • Thank for your suggestion @cheng .Let me check this possiblity – zen Jan 10 '18 at 06:25

1 Answers1

0

You could try to run your workload (currently the batch) with a threadpoolexecutor.

First setup a thread threadpoolexecutor. Then you need one thread that monitors the DB health (CPU, memory, whatever you find useful). Depending on the health it will increase or decrease the amount of threads in the threadpool.

Then you assign your workload as Runnables into the threadpoolexecutor, which will try to use all available threads to work off the queue of Runnables.

I know this approach does not use JSR-352 but at least you have control over the amount of parallel work. You may even think what should happen if you decrease the number of threads while all of them are in use. Should one of them be killed or should the threads reduce gracefully?

Queeg
  • 7,748
  • 1
  • 16
  • 42