1

Sample Code to explain my problem.

Sender

The Sender, that sends the request to the Server(over ZMQ) in format 

 ["sender-1", "sender-1-bdc1c14624076b691b8d9e15fbd754f1"]

 ..
 ["sender-99","sender-99-a2123d02c2989ef57da370bb13ba60e4"]

Server

The Server, upon receiving the data from sender relays it back to receiver in the same format.

  ["sender-1", "sender-1-bdc1c14624076b691b8d9e15fbd754f1"]

  ...
  ["sender-99","sender-99-a2123d02c2989ef57da370bb13ba60e4"]

Receiver

The Receiver, upon receiving the request from the Server just prints the message.

Description:

If I don't use a mutex defined in this line(inside the server). I see some data appearing at the receiver end which does not adhere to the above format/standard.

Example the Server would print (at this line)

"Sending sender-97 -- sender-97-9a284488454c8e8fd22bbbcf678895e9"

"Sending sender-98 -- sender-98-447eb5be94e7f6d949e764d7c88239ad"

But on the receiver end, I see messages that look like this.

sender-97 -- sender-98   
sender-98-22d9f01a8f801f559a9ff4e388813251 --

Question:

To me, it seems like a possible Thread issue(I may be wrong). Wherein the data that is passed to (inside Server) Celluloid (send) is getting changed by other thread.

I was under the impression that Celluloid handles almost all of your threads issue.

Is my understanding correct about all this?

Ratatouille
  • 1,372
  • 5
  • 23
  • 50

1 Answers1

0

You need an asynchronous logger.

Whenever you use puts you are outputting to a buffer which is actually very slow, compared to other things. What you ought to do, is use an actor used purely in the place of puts everywhere.

The best thing would be to instantiate a supervised actor, say named :logger and then interact with it inside your other actors by Celluloid[:logger].async.ouput() or else forward output to Celluloid[:logger].async so that every time you use output or console or something like that, it fires off an asynchronous method call. That way, even though your actor doing work is moving on to something else, the console output will still be in perfect sequence.

Your console output is being mangled because of the above issue with asynchrony in programming itself, not Celluloid ... and this is an extremely common problem with and without ZeroMQ.

Yes defer is disrupting the sequence of your Receiver output, but no this is not a Thread error in Celluloid. This is the natural behavior of asynchronous operations. In this case, you need to remove defer {} and keep async.received_data() like you have it.

Otherwise, as you see, Server will bombard Receiver with messages out of sequence. It doesn't help either that you're directly managing threads on top of all this, rather than letting Celluloid::ZMQ do it for you.

Also: I would remove all "external" thread management, and remove the defer {} wrapper... and let Celluloid::ZMQ smooth everything out, and keep sequence for you.

digitalextremist
  • 5,952
  • 3
  • 43
  • 62
  • @digitalexteemist what I dont a) understand the [`puts`](https://gist.github.com/meetme2meat/575fd1a712d47b60e6d8#file-server-rb-L44) mention above display the proper output but yet the data send over zeromq is different. b) if I remove defer from the code my code behave **sequentially** perhaps I forgot to mention the server call a external ruby code where all IO operation are perform something like [this](https://gist.github.com/meetme2meat/575fd1a712d47b60e6d8#file-server-rb-L39).c) if its not a thread issue how does using mutex actually solve this. – Ratatouille Mar 05 '16 at 02:03
  • Aplogize if you feel that the question are too vague or if they does make sense. Thanks for you help though. – Ratatouille Mar 05 '16 at 02:10
  • @digitalextermist I dont have any external thread the only place where the thread is used is in `sender`, which is done with an intent to test the concurrency of the `server` i.e whether the server can handle multiple requests at any given time.(Also please do read the comment 1 , point `b`) – Ratatouille Mar 05 '16 at 02:17
  • @Ratatouille the puts output is being done asynchronously, so it could appear out of order. The only way to know for sure is to use another actor which does output synchronously, even if it's used asynchronously. This is a very common problem with parallel and distributed systems – digitalextremist Mar 07 '16 at 02:43
  • @digitalextemist the puts is on the server end is displaying the correct value it's the output at the receiver end not (which prints the message over 0mq) are you referring to the receiver puts. – Ratatouille Mar 07 '16 at 03:05
  • Yes. Technically all log output ought to happen with an actor used by all three: sender, server, receiver – digitalextremist Mar 07 '16 at 03:20
  • The Logging actor can be instantiated by each one, doesn't need to be one they all use ( by 0MQ for example ) – digitalextremist Mar 07 '16 at 03:21
  • @digitalextermist I didn't understand the last comment.I 'll try to cross check the output witth ffi-rzmq libary only instead. – Ratatouille Mar 07 '16 at 06:04
  • The problem is not with `Celluloid` at all. My last comment was, you don't need to add a fourth "server" to the mix of sender/server/receiver, but merely make a logging actor used by those. I will update my answer. – digitalextremist Mar 07 '16 at 06:35
  • @digitalextremist. So would advise using mutex as a valuable approach if in case avoid data loss happening because of asynchronous programming. – Viren Mar 09 '16 at 06:07
  • @Viren in most cases, I would advise using `Celluloid` OR using `Mutex` ... almost always, using `Mutex` IN `Celluloid` means you're doing something wrong. There are cases where that's not true, but in most cases, if you use `Mutex` IN `Celluloid`, something is wrong. – digitalextremist Mar 09 '16 at 06:27
  • So, in Ruby, since I advocate for `Celluloid` I would say **NO** to `Mutex` ... but, I do use `Mutex` IN `Celluloid` ... in extreme cases only. Last resort cases. And this question is **NOT** even close to last resort. – digitalextremist Mar 09 '16 at 06:28
  • By the way, it's nice to see you @Viren. Do you use Twitter or GitHub? I would follow you in either place. – digitalextremist Mar 09 '16 at 06:29