0

Is there anybody who can explain me about Apache Ignite and Yardstick?

To my knowledge, Yardstick is a benchmark tool of Apache Ignite.

I know how to run Apache Ignite and how to set its memory capacity. (I know the default is 1 GB)

When I ran Ignite it shows:

[15:37:06] Topology snapshot [ver=1, servers=1, clients=0, CPUs=4, heap=4.0GB]

When I ran yardstick to benchmark it, Ignite shows:

[15:37:06] Topology snapshot [ver=2, servers=1, clients=1, CPUs=4, heap=6.0GB]

So why 2.0GB was increased? How can I modify it?

Also, this is what I know. Apache Ignite takes some amount of memory from the machine and using Yardstick, it can test the performance like cache.put() things within the memory took by Ignite. If it is correct then heap shouldn't be increased from 4.0GB to 6.0GB because Yardstick can benchmark within the memory (4.0GB)

Is there anybody who can explain me about this?

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Taejoon Kim
  • 1
  • 2

1 Answers1

2

Yardstick starts Ignite nodes with one of two roles: server or driver.

Drivers generate workload and measure performance and useful statistics. These are usually client nodes.

Servers form a core set of nodes, that process the workload, that is generated by drivers.

Each node is usually run in a separate process. Each one uses as much heap as you give it.

Looks like you started a server node manually and it has 4 GB of heap configured, and Yardstick driver has 2 GB. That's how 4+2 gives 6 :)

P.S. If you use shell scripts, distributed with Ignite binaries, to run benchmarks, you can specify number of nodes and their configurations in a property file. Yardstick will start server nodes itself, and you won't have to do it manually.

Here is an example of such property file with comments for all configured properties: https://github.com/apache/ignite/blob/master/modules/yardstick/config/benchmark-atomic.properties

Denis
  • 3,573
  • 8
  • 13
  • Can you explain me more? So I manually set the server node with 4 GB, then it will occupy 4 GB in memory then what does it do? I just imagined that to set the heap size is "Hey memory, I'm a server node of Apache Ignite, I'm going to use 4 GB in you and my driver nodes will use the memory in which I occupy" But it uses 4 GB heap size by the server node and 2 GB heap size by the driver node. Do you know what exactly mean? – Taejoon Kim May 21 '18 at 20:55
  • Drivers and servers are different Java processes. You can't make different processes share the same heap. So, servers and drivers work with their own memory space. – Denis May 22 '18 at 07:38
  • Okay Thanks!! It help me! – Taejoon Kim May 22 '18 at 13:24