0

I have multiple processes running on multiple machines using log4j (2.11). I need to consolidate the logging messages to be displayed on the front-end, and would like each process to use a ZeroMQ Appender to publish the log messages to a single connection. I will then have one subscriber receiving the messages, performing the consolidation, and then displaying the log messages.

I have a toy application working with one publisher (process logging); however, when multiple processes try to connect to the same endpoint, I receive the "Address already in use" error message. Which (most likely) means the log4j ZeroMQ (JeroMQ) appender is doing the "binding," since only one process can bind a zmq socket.

Is there a configuration option to have the log4j ZeroMQ Appender perform a "connect" instead of the "bind" or is there another option available to accomplish the same goal.

Walter
  • 11
  • 2
  • You may be well aware, there is no benefit in using `PUB/SUB` archetype, unless your infrastructure is willing to operate many-`PUB`-s + many-`SUB`-s, as if building an `N+M`-fault-resilience into your loging service. An ordinary `PUSH/PULL` would fit better, as there is Zero-need for subscription based TOPIC-filtering in doing plain logging, isn't it? ( BTW, is the **`2.11` above** referring to the ZeroMQ native API version? Many features now present in native API v4.2+ are not present there, so quite important to be aware of that ) – user3666197 Jun 29 '18 at 04:40
  • The `2.11` is referring to the version of Log4J I am using (currently the latest) which offers a ZeroMQ (JeroMQ) Appender. Ideally, the subscriber will receive the logging messages without having to poll the other processes. I'll read on the `PUSH/PULL` architecture to see if that would be a better fit; however, with the `PUB/SUB` I can receive all of the messages without having to subscribe to each individual topic. So, unless I'm misunderstanding something, the TOPIC-filtering will not be a concern. – Walter Jun 29 '18 at 16:51

1 Answers1

0

There's two possible reasons for "hardware" rejected .bind()-s

One has been already in your post -- under an assumption, that the JeroMQ-services are indeed trying to rigidly .bind() in places, where you would like to .connect() instead. Review the code-base to either confirm that ( with a possible fork/mod circumventing any such discomfort in a refactored service fashion ) or reject this alternative, if the code is not the one to blame.

But let me mention another possible reason, which was quite often a problem during PoC / prototyping.

Early-stage mock-up tools are prone to crashes ( and yes, sometimes crash peacefully, but sometime do crash more times or harder than the PoC team is willing to live with ).

If the code-base is not ( yet ) well designed during this phase to clearly handle also the resources management ( proper use of explicit aSocket.setsockopt( LINGER, 0 ) ( a lethal must in native API versions pre-4.x ) {aSocket|aMessage}.close() and Context.term() methods ) that does a clean exit even after a crash, there are situations, that your code has left a still active ( not dismantled ) one or more Context()-instances, which ( still ) block the hardware resources, as it were not allowed to release them yet.

This has sometimes led to a need to reboot a platform, as a deadlocked Context()-instance has no other way to get "vacuum-cleaned" ( shame on those, who do not fuse/protect the code for surviving till a final stage in try: except: finally: triade, to enforce a graceful termination and clean release of all allocated resource ).

halfer
  • 19,824
  • 17
  • 99
  • 186
user3666197
  • 1
  • 6
  • 50
  • 92
  • I did confirm that a `.bind()` was being used in the `JeroMQAppender`, and I do not see an option in the code to choose a `.connect()` instead. So I am in the process of modifying the `JeroMQAppender` to allow the option of performing a `.connect()` rather than a `.bind()`. – Walter Jun 29 '18 at 17:02
  • @Walter Ok, understood that you have chosen to go into the option **one**. So, G/L in re-factoring to get the one `.bind()` and many `.connect()`-s working smooth and sound. – user3666197 Jun 29 '18 at 17:24