3

I'm developing a peer to peer message parsing application. So one peer may need to handle many clients. And also there is a possibility to send and receive large data (~20 MB data as one message). There can be situations like many peers send large data to the same peer. I heard there are many solutions to handle these kind of a situation.

  • Use thread per peer
  • Using a loop to go through the peers and if there are data we can recive
  • Using select function
  • etc.

What is the most suitable methodology or most common and accepted way to handle these kind of a situation? Any advice or hint are welcome.

Updated: Is there a good peer to peer distributed computing library or framework for C++ on windows platform

Amith Chinthaka
  • 1,015
  • 1
  • 17
  • 24
  • Related questions: [Multiple Socket client connecting to a server](http://stackoverflow.com/questions/5468519/multiple-socket-client-connecting-to-a-server) – Amith Chinthaka Feb 20 '14 at 06:38

2 Answers2

2

Don't use a thread per peer; past the number of processors, additional threads is likely only to hurt performance. You'd also have been expected to tweak the dwStackSize so that 1000 idle peers doesn't cost you 1000MB of RAM.

You can use a thread-pool (X threads handling Y sockets) to get a performance boost (or, ideally, IO Completion Ports), but this tends to work incredibly well for certain kinds of applications, and not at all for other kinds of applications. Unless you're certain that yours is suited for this, I wouldn't justify taking the risk.

It's entirely permissible to use a single thread and poll/send from a large quantity of sockets. I don't know precisely when large would have a concernable overhead, but I'd (conservatively) ballpark it somewhere between 2k-5k sockets (on below average hardware).

The workaround for WSAEWOULDBLOCK is to have a std::queue<BYTE> of bytes (not a queue of "packet objects") for each socket in your application (you populate this queue with the data you want to send), and have a single background-thread whose sole purpose is to drain the queues into the respective socket send (X bytes at a time); you can use blocking socket for this now (since it's a background-worker), but if you do use a non-blocking socket and get WSAEWOULDBLOCK you can just keep trying to drain the queue (here it won't obstruct the flow of your application).
Mr. Smith
  • 4,288
  • 7
  • 40
  • 82
  • You mean data to be sent should first queue and then another function (may be a function which runs in a separate thread) should send these data one by one (while doing wait on block and so and so). Right? Yeah it seems a solution. Then data will be delivered to the end but there is no guarantee of the time it will spend. If I need to deliver data within a time limit (a fair limit), what are the techniques which I can follow? – Amith Chinthaka Feb 20 '14 at 07:38
  • @AmithChinthaka If you'd queued 20MB to be sent out to a peer, and then decide you need to send `4KB` urgently (you can't wait for the 20MB to drain) than that is quite the tricky challenge! You can ensure stuff is sent fairly (each peer given equal drain priority) with this design, but not much past that. – Mr. Smith Feb 20 '14 at 07:53
  • hmm.. Are these kind of blocking, waiting and queuing things general scenarios with the network communication or I just get these since I'm doing something wrong? – Amith Chinthaka Feb 20 '14 at 07:59
  • @AmithChinthaka I'm afraid so; it's an advanced subject. You're not doing something wrong. – Mr. Smith Feb 20 '14 at 09:24
0

You could use libtorrent.org which is built on top of boost (boost-asio ). It's focusing on efficiency and scalability.

I have not much experience in developing a socket in C++ but in C# I had really good experience accepting connections asynchronously and pass them to an own thread from a threadpool.

Andre Hofmeister
  • 3,185
  • 11
  • 51
  • 74
  • Yeah I'm aware of libtorrent. But it's a bittorrent implementation. I'm not going to send files. But the data which I planing to send over the network can vary 1 KB to maximum 20 MB. So using bittorrent means additional cost to me. Sharing trackers and so many stuffs. Anyway thanks. – Amith Chinthaka Feb 20 '14 at 06:48
  • 1
    Then I would reommend to use a non-blocking, asynchronously socket. _[Boost.Asio](http://www.boost.org/doc/libs/1_43_0/doc/html/boost_asio.html) is a cross-platform C++ library for network and low-level I/O programming that provides developers with a consistent asynchronous model using a modern C++ approach._ Are the files which one peers receives equal to the other? – Andre Hofmeister Feb 20 '14 at 06:57
  • I'm using non-blocking sockets. But the problem is when one peer send data continuously to a socket, it blocks temporary and returns `WSAEWOULDBLOCK` error. This happen because of write file descriptor of that socket may be filed. – Amith Chinthaka Feb 20 '14 at 07:03
  • I guess you have to wait until the buffer space is free. _WSAEWOULDBLOCK is not really an error but simply tells you that your send buffers are full. This can happen if you saturate the network or if the other side simply doesn't acknowledge the received data._ Have a look at [this](http://stackoverflow.com/questions/14546362/how-to-resolve-wsaewouldblock-error). – Andre Hofmeister Feb 20 '14 at 07:05
  • Yeah that is what I'm doing currently. But when many peers (i.e. 4 or 5) send data (4 MB) to a same socket at same time, the wait time is high. Is there any better solution. – Amith Chinthaka Feb 20 '14 at 07:12
  • Then you're implementation of a non-blocking (asynchronous) socket is wrong, I guess. This never happens to me. – Andre Hofmeister Feb 20 '14 at 07:16