I am getting a heap memory issue while running DSBULK load

Question

I have unloaded more than 100 CSV files in a folder. When I try to load those files to cassandra using DSBULK load and specifying the the folder location of all these files, I get the below error

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "unVocity-parsers input reading thread"

I wanted to see if anyone else has faced it and how it has been resolved.

In the logs, it shows allocated is 1739 MB. Is it possible to mention a different value when running DSBULK — Rajib Deb, Jan 24 '22 at 20:07

score 2 · Accepted Answer · answered Jan 24 '22 at 20:24

2

Here are a few things you can try:

You can pass any JVM option or system property to the dsbulk executable using the DSBULK_JAVA_OPTS env var. See this page for more. Set the allocated memory to a higher value if possible.
You can throttle dsbulk using the -maxConcurrentQueries option. Start with -maxConcurrentQueries 1; then raise the value to get the best throughput possible without hitting the OOM error. More on this here.

answered Jan 24 '22 at 20:24

adutra

4,231
1
20
18

Thanks, i am following #2 and it is working. – Rajib Deb Jan 25 '22 at 05:18

I am getting a heap memory issue while running DSBULK load

1 Answers1