0

I am new to Infinispan. We have setup a Infinispan cluster so that we can make use of the Distributed Cache for our CPU and memory intensive task. We are using UDP as the communication medium and Infinispan MapReduce for distributed processing. The problem we are using facing is with the throughput. When we run the program on a single node machine, the total program completes in around 11 minutes i.e. processing a few hundred thousand records and finally emitting around 400 000 records.

However, when we use a cluster for the same dataset, we are seeing a throughput of only 200 records per second being updated between the nodes in the cluster, which impacts the overall processing time. Not sure which configuration changes are impacting the throughput so badly. I am sure its something to do with the configuration of either JGroups or Infinispan.

How can this be improved?

James Z
  • 12,209
  • 10
  • 24
  • 44
Vijay Bhoomireddy
  • 576
  • 8
  • 19
  • I'm only a little familiar with how to configure Infinispan, but what are your system's write/read buffer sizes? Ex: `net.core.rmem`/`net.core.wmem` on Linux. Maybe they should be increased. This can impact UDP performance. – mdnghtblue Nov 19 '14 at 04:04
  • We have increased the buffer sizes to 200MB at the operating system level in order to match the ones provided in UDP properties in the jgroups xml file. However, we havent seen any performance gains. – Vijay Bhoomireddy Nov 19 '14 at 04:20
  • What Infinispan version are you using? How big are your records? – Dan Berindei Nov 19 '14 at 21:25
  • Dan, we are using version 6.0.2. As its a just test dataset, records are not such big ones. They contain roughly around 10-35 columns across datasets. Each columns could be 40 chars long – Vijay Bhoomireddy Nov 20 '14 at 05:00
  • Could you crosslink this question with the one on JBoss Forum? – Radim Vansa Nov 20 '14 at 08:12

0 Answers0