Multiprocessing vs gevent

Question

Currently I am using zeromq with pub-sub pattern, I have single worker to publish and many(8) subscriber (all will subscribe) to same pattern.

Now I tried multiprocessing to spawn subscribers it works. I am missing few messages.

Why I am using multiprocessing is to handle each message as it arrives and process them, every second publisher publishes 10 - 100 messages.

In this case it is advised use multiprocessing or gevent ?

score 4 · Answer 1 · answered Apr 03 '12 at 21:51

4

Multiprocessing will clearly have much higher memory overhead but will utilize another core (and you don't have to worry about lack of pre-emption) so.. it depends on your needs. It's likely that multiple processes using gevent will get you the highest throughput / lowest latency.

answered Apr 03 '12 at 21:51

tmc

716
7
15

So you say that multiprocessing let us not worry about pre-emption please? – Avv Oct 11 '22 at 19:58

score 4 · Answer 2 · answered Apr 18 '12 at 09:10

4

I am answering my own question

I choose multiprocessing over gevent

Server had 8 cores
Parallelism was needed
Choose ZMQ over multiprocessing queue or gevent queue.

answered Apr 18 '12 at 09:10

Kracekumar

19,457
10
47
56

So i guess the processes num are fixed, equal to the core num. – hunter_tech Nov 20 '19 at 08:01
I am not sure about the points you mentioned under multiprocessing? – Avv Oct 11 '22 at 19:55

Multiprocessing vs gevent

2 Answers2

I am answering my own question

Linked