-1

I want to split up the following workflow of a C++ program:

Read serialized data (1 sec, which is already very fast for that size)
Search data (0.01 ms)
Return found data(0.00.. ms) Edit: found data is just a small file

Obviously reading the data takes most of the time. Additionally, the data that is being read, never changes. To speed up the workflow, I want to read the data once, then wait for requests. Just like a MySQL db, that is waiting for statements. Or any other Process that is running and just waiting for something to do. Right now, I start the program using the shell, and it will only be run inside a linux docker container.

What would be your approach to do that?

Goal: The program is always running and waiting for search metadata from another process (bash script?, a nodejs exec call), then perform search, then return data (write to file)

Dependency: I want to build this without eg nodejs v8 integration

Is Boost Asio a good startingpoint? Do I need to build a socket server? Are there easier ways?

schlumpel
  • 174
  • 1
  • 8
  • 1
    Your question suggests, but does not explicitly state, that the serialized data may not be a local file but is instead read from a network connection. Can you please confirm? – user4581301 May 12 '22 at 20:15
  • _"the data that is being read, never changes"_ - and yet it seems to be quite a big chunk of data. If it never changes, why send it all over and over again? – Ted Lyngmo May 12 '22 at 20:19
  • Yeah you are right, the data is a local serialized file. Because it is a big chunk of data i do NOT want to load it again and again for each startup of the program. I want to keep the program running, waiting for requests. Like a while(1=1) loop after the data has been loaded. – schlumpel May 12 '22 at 21:05
  • Edited the question: By return found data i meant returning like a small piece of the perviously loaded data. Just a small file. That was confusing, I'm sorry! – schlumpel May 12 '22 at 21:16

2 Answers2

0

This is very opinion-based.

I'd turn the program into a REST server. I hate boost, though, and recommend Poco::Net instead. If you want to see an example implementation:

git@github.com:jplflyer/ShowLib.git

And look at the RESTServer class.

I use JSON to pass messages around. It's become the industry standard.

Joseph Larson
  • 8,530
  • 1
  • 19
  • 36
  • Of course this is very opinion-based, and thats why I'm very interested in your opinion :-) Thank you for the answer! As for now, all REST servers i've worked communicated with the web. My program will just receive requests from the local system. Does that information change your suggestion? – schlumpel May 12 '22 at 21:09
  • If you want to load the data just once, then your program needs to stay up with the data in memory, which means you need some way to communicate with it. REST means you're not writing socket code yourself. I think it's way easier to code than anything else you might do. – Joseph Larson May 12 '22 at 22:16
  • After doing more research, I think shared memory IPC is what I’m looking for. – schlumpel May 16 '22 at 04:28
0

I've used Boost.ASIO for network communication and it is a very good library with good documentation and example code.

But Boost is a huge library. One good thing about Boost.ASIO is you can compile it as a standalone library without much of the boost headers. Have a look here on how to compile Boost.ASIO as standalone library.

Aamir
  • 1,974
  • 1
  • 14
  • 18