0

I am new to Hazelcast. When Imap filter with predicate is triggered on a HazelCast Jet pipeline, in a VDI I see the porcessors count is only 2 and on my machine I see 12 porcessors.

When there is only 2 processor, filter job is completing very fast. But when I see 12 processors on my machine, filter job keeps running for longer time and keeps logging a line of Hazelcast healthmonitor log and times out. I assume that this is the problem.

From my search, I understand that Hazelcast create number of processors based on CPU. How can I control the this processor count.

Below is the line of log when Hazelcast jet client is created and IMap filter is applied on a pipeline.

Pardon me, I couldn't give clear description of issue with limited understanding of Hazelcast.

Jun 08, 2021 9:31:20 PM com.hazelcast.internal.diagnostics.HealthMonitor
INFO: [*xxx.xx.xxx.xxx*]:5701 [jet] [3.1] **processors=12**, physical.memory.total=32.0G, physical.memory.free=224.8M, swap.space.total=2.0G, swap.space.free=716.5M, heap.memory.used=110.4M, heap.memory.free=182.1M, heap.memory.total=292.5M, heap.memory.max=7.1G, heap.memory.used/total=37.74%, heap.memory.used/max=1.52%, minor.gc.count=4, minor.gc.time=75ms, major.gc.count=2, major.gc.time=129ms, load.process=3.01%, load.system=93.62%, load.systemAverage=9.20, thread.count=78, thread.peakCount=78, cluster.timeDiff=0, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.client.query.size=0, executor.q.client.blocking.size=0, executor.q.query.size=0, executor.q.scheduled.size=0, executor.q.io.size=0, executor.q.system.size=0, executor.q.operations.size=0, executor.q.priorityOperation.size=0, operations.completed.count=392, executor.q.mapLoad.size=0, executor.q.mapLoadAllKeys.size=0, executor.q.cluster.size=0, executor.q.response.size=0, operations.running.count=0, operations.pending.invocations.percentage=0.00%, operations.pending.invocations.count=2, proxy.count=8, clientEndpoint.count=0, connection.active.count=0, client.connection.count=0, connection.count=0
logi-kal
  • 7,107
  • 6
  • 31
  • 43
Parthi
  • 123
  • 1
  • 7

1 Answers1

0

The processors=12 is a red herring I guess - it's just the value of Runtime.availableProcessors().

From my search, I understand that Hazelcast create number of processors based on CPU

By this you probably mean the number of Jet parallel processors. You can control the number of parallel processors by setting localParallelism(n) in your Pipeline.

A line from Health Monitor is logged when there's unusual load on the system. It can be caused by some other process in the system maxing out the CPU. Your question doesn't give enough information to guess.

Oliv
  • 10,221
  • 3
  • 55
  • 76
  • Thanks @Oliv. Since I am just working directly on a project which is already using Hazelcast, I didn't get chance to understand complete working. Is there some setting in the HazelCast client config or jet config to make the map filter faster. As I mentioned above, which I make the request from windows VDI machine with less processing I always see only 2 processor. But when I run it on local with high processing processing speed, I see this 12 processor. Always it works on VDI and it always fails on local. Any suggestions. – Parthi Jun 09 '21 at 08:41
  • Running the same code on two different machines and this is what I see, ***Faster Machine*** `Jun 07, 2021 4:53:17 PM com.hazelcast.jet.impl.JetService INFO: [xxx.xx.xxx.xx]:5702 [jet] [3.1] Setting number of cooperative threads and default parallelism to 2` ***Slower MAchine*** `Jun 09, 2021 11:22:06 PM com.hazelcast.jet.impl.JetService INFO: [xxx.xx.xxx.xx]:5701 [jet] [3.1] Setting number of cooperative threads and default parallelism to 12` – Parthi Jun 09 '21 at 17:55
  • I also see this difference from logs on pc where query is faster `hz._hzInstance_1_jet.partition-operation.thread-1` and on pc where query is slow `hz._hzInstance_1_jet.jet.cooperative.thread-7`. – Parthi Jun 09 '21 at 18:16
  • @Parthi the cooperative thread count is specified in the `hazelcast-jet.yaml` file. If it's not there, it uses `Runtime.availableProcessors()`. – Oliv Jun 10 '21 at 12:13
  • I see one request using cooperative thread and other using non cooperative. How does that happen? – Parthi Jun 17 '21 at 04:51