0

I'm working on a project where I'm required to make 2 processes (one written in C++ and the other in Python) talk asynchronously. Specifically they need to:

  • Never use any kind of wait or sleep
  • I have no way to determine whether one of the 2 processes will be ready to write or read
  • Multithreading is not allowed
  • Both are running in loop, so at the beginning of the loop they should call either a read or write function and viceversa when they reach the end of the loop
  • Communication needs to be resilient: if the C++ process writes while the Python one is crashing, the data should be sent regardless and the Python process should retrieve said data after being rebooted
  • The data sent won't be as simple as just sending signals. It's gonna be strings etc
  • Zero tolerance for data loss
  • Shared memory is off the table unless it's literally the ONLY available solution

At first I've been looking at FIFOs, however they seem to be unreliable: the resiliency is just not there (which is strange because it works when I try to make 2 C++ processes talk to each other while one of them has crashed) and many strings my C++ process is sending to the Python one are just completely lost for some reason. Finally, 2 way communication doesn’t seem to work at all unless I create 2 FIFOs, since the same process could read and write before the other has a chance to read

Now I'm looking at Unix Domain Sockets, but they seem to be used for synchronous communication only.

I'd need some guidance to understand whether I'm misusing the FIFOs (aka: FIFOs are actually compliant with my requests and I'm just doing something wrong) or UDS are a viable option or there are actually other available solutions.

someCoder
  • 185
  • 3
  • 15
  • Can you use an existing message broker such as rabbitmq, redis, kafka, etc? – jordanm Jun 17 '20 at 15:13
  • 1
    (1) FIFOs are definitely reliable, the fact that it doesn't work between a C++ and Python application indicates there is a bug in your code. (2) Unix sockets can be used asynchronously, like any socket. (3) This question is extremely broad, so you'll need to try something and then post a question when you run into problems. – bnaecker Jun 17 '20 at 15:13
  • @jordanm we use kafka for other stuff but cannot use it for this specific issue – someCoder Jun 17 '20 at 15:14
  • @bnaecker ok, I'll guess I'll try to spend some more time with the FIFOs. I'll leave the question open for a while just to see if someone else comes up with something otherwise I'll close it and follow your suggestion (3) – someCoder Jun 17 '20 at 15:15
  • 1
    FIFOs (and pipes) are one-way, so you are correct that you will need two of them: one to send data from A->B the other from B->A. – bnaecker Jun 17 '20 at 15:16

0 Answers0