0

I have a simple python server script which forks off multiple instances (say N) of C++ program. The C++ program generates some events that need to be captured.

The events are currently being captured in a log file (1 logfile per forked process). In addition, i need to periodically (T minutes) get the rate at which the events are being produced across all child processes to either the python server or some other program listening for these events (still not sure). Based on rate of these events, some "re-action" may be taken by the server (say reduce the number of forked instances)

Some pointers i have briefly looked at:

  • grep log files - go through the running process log files (.running), filter those entries generated in the last T minutes, analyse the data and report
  • socket ipc - add code to c++ program to send the events to some server program which analyses the data after T minutes, reports and starts all over again
  • redis/memcache (not sure completely) - add code to c++ program to use some distributed store to capture all the generated data, analyses the data after T minutes, reports and starts all over again

Please let me know your suggestions.

Thanks

harish
  • 109
  • 2
  • 6
  • Voting to migrate to Programmers.SE -- by definition, anything asking for suggestions rather than concrete answers does not belong on SO. – ildjarn Jan 10 '12 at 00:01

2 Answers2

0

As a alternative to your socket IPC suggestion, how about 0mq. It's a library (in C with python bindings available) that can do message transfer on an inter-thread, inter-process or inter-machine level. Pretty simple to get going, and pretty quick.

I'm not affiliated with it. I'm just evaluating it for other uses and thought it might be a fit for you as well.

Paul S
  • 7,645
  • 2
  • 24
  • 36
0

if time is not of the essence (T minutes sounds like it is long compared to whatever events are happening in the C++ programs that are kicked off) then dont make things any more complicated than they need to be. forget IPC (sockets, shared mem, etc), just have each C++ program log what you need to know about time/performance and let the python script check logs every T minutes that you need the data. dont waste time overcomplicating something that you can do in a simple manner

djb
  • 16