0

I am benchmarking cratedb with each record having 40 columns and record size varies between 300bytes to 1kb.

I am able to get only 500 inserts per second on single thread.

Using cratedb 1.0.1 with python client. Machine having 16 CPUs and 32GB RAM.

Please help how can I increase the performance. And on what conditions we get 40,000 inserts per second mentioned in official website.

Thanks in Advance

Carbonrock
  • 457
  • 2
  • 15

1 Answers1

1

40k inserts a second are doable with CrateDB and should not be a huge problem. However, in a single thread the performance is probably lower and depending on response times. If possible, it's recommended to use a multithreaded insert procedure (or at least asynchronous), since CrateDB can handle quite a few concurrent connections. The 40k were achieved with more than a single thread issuing bulk requests.

So the performance issue you are experiencing can have a few causes:

  • Your machine doesn't have an SSD (SSDs are crucial for the best performance)
  • The table's shard count is too high/too low
  • The CRATE_HEAP_SIZE environment variable is not set correctly
  • Your bulk size could be higher

Can you check all of those?

Cheers, Claus

claus
  • 377
  • 2
  • 9