1

Recently I have come across the POCO and ACE networking frameworks, along with Boost which I already knew about.

My question is, are these libraries any faster for passing messages than just a regular C program with Berkeley sockets? Are these libraries only popular because they add in the features of multi-threading etc, which helps the performance factor?

I want to write a high-performance messaging system on Linux, but I cant work out if I should avoid ACE , POCO and Boost and instead just using the Linux thread OS functions along with berkeley sockets?

In other words, I am not bothered about generic code, making my code "STL friendly" etc. I just want raw performance (without having to write assembly!).

user997112
  • 29,025
  • 43
  • 182
  • 361

3 Answers3

1

Have you looked at 0MQ (aka ZeroMQ) yet? Quoting from their website:

ØMQ \zeromq\:
 Ø  The socket library that acts as a concurrency framework.
 Ø  Faster than TCP, for clustered products and supercomputing.
 Ø  Carries messages across inproc, IPC, TCP, and multicast.
 Ø  Connect N-to-N via fanout, pubsub, pipeline, request-reply.
 Ø  Asynch I/O for scalable multicore message-passing apps.
 Ø  Large and active open source community.
 Ø  30+ languages including C, C++, Java, .NET, Python.
 Ø  Most OSes including Linux, Windows, OS X.
 Ø  LGPL free software with full commercial support from iMatix.
Dirk Eddelbuettel
  • 360,940
  • 56
  • 644
  • 725
  • Hi, thank you very much for this. I haven't heard of this before! – user997112 Dec 03 '11 at 21:25
  • It's pretty neat, I've following it for a few years. An earlier version used to have the two-machine throughput test as an example, but that may have been withdrawn. There are also a number of neat whitepapers on their site. And it is already being used by a number of places/projects. – Dirk Eddelbuettel Dec 03 '11 at 21:27
  • Hey Dirk, out of curiousity do you know much about the architecture? I presume it must be an underlying C architecture? – user997112 Dec 03 '11 at 22:47
  • Do you have a link? google isnt showing up results for trying to find the source code – user997112 Dec 04 '11 at 00:02
  • Try a Google search for "site:github.com zeromq" – Dirk Eddelbuettel Dec 04 '11 at 00:13
0

Disclaimer: I wrote it.

If you are looking for raw speed, especially with multiple subscribers for a signal, and simple integration, we tried to solve that problem with DSTC (Distributed C).

Upsides

  • One header file to include, two libraries to link.
  • No stub code generation or weird build steps.
  • One line of C code to export or import a remote function.
  • 10M RPC calls / second over a 10GB Ethernet, using a single core.
  • 20M calls / sec over localhost on a decent laptop.
  • C/C++/Python support.

Downsides:

  • No security, it must run in a protected environment.
  • Primitive. It does only two things, RPCs and pub/sub.
  • No return values, although callbacks are available.
notatallswedish
  • 21
  • 4
0

You are asking a lot without sharing much about your use case. From what you write, it seems you need to do messaging between machines connected to a network. Is it all peer to peer, or are some machines "servers", and others clients? Libraries such as ACE offer everything from simple convenience classes for wrapping any type of socket communication and multithreading, up to full blown servers, using virtually any model you can think of.

Assuming you need some kind of servers, there's the whole debate about whether you should go with threading, or single threaded async. Again, depending on the use case, one will be "better" than the other (it really depends on what you need to do).

On the network part, do you need reliable ordered messages, or is each message obsolete by the time you would be able to detect that a packet got lost? For reliability you would typically build on top of TCP, but depending on what you need to do, you could design a protocol on UDP which will run faster.

Unless you requirements are really simple and basic, and/or you've written lots of networked multithreaded code before, you are most likely better off to use a well written package than trying to reinvent everything on your own.

Marius Kjeldahl
  • 6,830
  • 3
  • 33
  • 37
  • Apologies, I am doing this for teaching myself- as opposed to having a certain scenario in mind. I was going to have a machine which I will benchmark how many messages it can receive per second, being sent from another two different machines. – user997112 Dec 03 '11 at 21:23
  • With regards to ordering, I am not sure: I want to send FIX (http://en.wikipedia.org/wiki/Financial_Information_eXchange) messages and my aim will be to try different systems which can capture the most FIX messages per second. I guess because I am simulating electronic trade orders I will care if they get lost, so TCP? – user997112 Dec 03 '11 at 21:24
  • Still, you need to figure out what you need to do with the messages received. That will probably dictate suitable architectures for your setup. If you just need to receive lots of TCP packets, async single threaded will probably give you most speed for the buck, but once you need to do things with those packets, the picture becomes less clear. – Marius Kjeldahl Dec 03 '11 at 21:29
  • is that because once I begin to do something with the packets, a bottleneck occurs? Meaning, as soon as I begin to process the packets which have already arrived, I am neglecting the packets which are still arriving and the arriving ones will begin to accumulate? – user997112 Dec 03 '11 at 22:46