0

I'm using the following in my Java client to connect to a local instance of Ignite:

    IgniteConfiguration cfg = new IgniteConfiguration();
    cfg.setClientMode(true);
    cfg.setIgniteInstanceName(getInstanceName());
    cfg.setPeerClassLoadingEnabled(true);

    TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
    ipFinder.setAddresses(getIpFinderAddresses());
    cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));

    return Ignition.start(cfg);

If the server instance is not running, my (spring-boot) application starts flooding the logs with exceptions and thread stack dump, which I found very annoying and useless.

For example:

2021-10-05 10:34:50.781 ERROR 5548 --- [alance-service%] o.apache.ignite.internal.util.typedef.G  : Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [workerName=tcp-client-disco-msg-worker, threadName=tcp-client-disco-msg-worker-#4%my-service%-#54%my-service%, blockedFor=14s]
2021-10-05 10:34:50.791  WARN 5548 --- [alance-service%]                                          : Possible failure suppressed accordingly to a configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=tcp-client-disco-msg-worker, igniteInstanceName=my-service, finished=false, heartbeatTs=1633422875964]]]
org.apache.ignite.IgniteException: GridWorker [name=tcp-client-disco-msg-worker, igniteInstanceName=my-service, finished=false, heartbeatTs=1633422875964]
  at java.base@11.0.11/java.net.PlainSocketImpl.waitForConnect(Native Method) ~[na:na]
  at java.base@11.0.11/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[na:na]
  ...
2021-10-05 10:34:50.792  WARN 5548 --- [alance-service%] o.a.i.i.p.failure.FailureProcessor       : No deadlocked threads detected.
2021-10-05 10:34:50.868  WARN 5548 --- [alance-service%] o.a.i.i.p.failure.FailureProcessor       : Thread dump at 2021/10/05 10:34:50 CEST
  ...
  ... hundreds of lines with thread dump (why one would want to see them if - as stated above - there were no deadlocks?) ...
  ...
2021-10-05 10:34:57.436  WARN 5548 --- [alance-service%] o.a.i.spi.discovery.tcp.TcpDiscoverySpi  : Failed to connect to any address from IP finder (will retry to join topology every 2000 ms; change 'reconnectDelay' to configure the frequency of retries): [/127.0.0.1:47500, /127.0.0.1:47501, /127.0.0.1:47502, /127.0.0.1:47503, /127.0.0.1:47504, /127.0.0.1:47505, /127.0.0.1:47506, /127.0.0.1:47507, /127.0.0.1:47508, /127.0.0.1:47509]

And of course this is printed every 5-10 seconds, while the only line I'm interested in is the last one (Failed to connect to any address from IP finder)

I can I configure Ignite to handle such logging, or to make it silently accept the fact that the server is not running?

Jack
  • 1,488
  • 11
  • 21

1 Answers1

1

I believe in your case the most verbose message comes from the FailureProcessor. It prints a full thread dump when a failure is detected. There are two system properties that control this behaviour.

  1. IGNITE_DUMP_THREADS_ON_FAILURE - totally disables thread dumps, it's better to leave it as true because it could be really useful in case of other failure types.
  2. IGNITE_DUMP_THREADS_ON_FAILURE_THROTTLING_TIMEOUT - this one sets a minimal gap (in millis) between two messages with thread dumps.

Here are javadocs: IGNITE_DUMP_THREADS_ON_FAILURE and IGNITE_DUMP_THREADS_ON_FAILURE_THROTTLING_TIMEOUT.

Vladimir Pligin
  • 1,547
  • 11
  • 18
  • Thansk, this helps for thread dump logs. Any chance to handle the exception part (which of course is less verbose but still annoying)? – Jack Oct 05 '21 at 11:27