10

having several connections in several different threads.. I'm basically doing a base class that uses boost/asio.hpp and the tcp stuff there.. now i was reading this: http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/tutorial/tutdaytime1.html it says that "All programs that use asio need to have at least one io_service object." so should my base class has a static io_service (which means there will be only 1 for all the program and a all the different threads and connections will use the same io_service object) or make each connection its own io_service?

thanks in front!

update: OK so basically what I wish to do is a class for a basic client which will have a socket n it. For each socket I'm going to have a thread that always-receives and a different thread that sometimes sends packets. after looking in here: www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/reference/ip__tcp/socket.html (cant make hyperlink since im new here.. so only 1 hyperling per post) I can see that socket class isn't entirely thread-safe..

so 2 questions: 1. Based on the design I just wrote, do I need 1 io_service for all the sockets (meaning make it a static class member) or I should have one for each? 2. How can I make it thread-safe to do? should I put it inside a "thread safe environment" meaning making a new socket class that has mutexes and stuff that doesn't let u send and receive at the same time or you have other suggestions? 3. Maybe I should go on a asynch design? (ofc each socket will have a different thread but the sending and receiving would be on the same thread?)

just to clarify: im doing a tcp client that connects to a lot of servers.

grich
  • 463
  • 3
  • 6
  • 14
  • "at least one io_service" means "one or more". – Abyx Nov 02 '10 at 14:14
  • my question was what should i do and why? only 1 for all or one for each and why? – grich Nov 02 '10 at 14:15
  • Are you doing synchronous or asynchronous io? – Inverse Nov 02 '10 at 14:20
  • i use another thread to receive and put it all in a thread-safe buffer if that's your question – grich Nov 02 '10 at 14:23
  • @grich do you use the `async_read` and `async_write` or `read` and `write` free functions? – Sam Miller Nov 02 '10 at 16:04
  • I want to use read and write... But my third question was if it would be better to do it in an asynch way? – grich Nov 02 '10 at 16:12
  • @grich I strongly suggest looking into using the asynchronous methods, that is really where the powerfulness of the Boost.Asio library lies. – Sam Miller Nov 02 '10 at 16:16
  • @Sam Miller I have looked into it and it doesn't seem to be very relevant to what i asked/intend to do. Neither does it answer my question tbh. I still need a different thread for receiving and sending since i want it to happen at the same time (me sending data has nothing to do with the server send data to me since the server can send me data out of the blue and I have to read it right away, cant wait for me to do io_service.run()). I still cant figure a way to do this operations in parallel (receiving and sending). neither do I know how it would effect the num of sockets and io_service – grich Nov 02 '10 at 16:38
  • @grich I don't think you understand how asynchronous operations work. In a nutshell, you post operations to the `io_service` queue. When those operations are completed, the `io_service` invokes your handler, which can then post additional asynchronous operations. There is no waiting for your application to invoke `io_service::run`. Your application does whatever setup it needs to, invokes `io_service::run` from one or more threads, then the async handlers take over. There are plenty of examples of this paradigm on the Boost.Asio website, I suggest you take a look at them. – Sam Miller Nov 02 '10 at 16:51
  • @Sam Miller I did understand that and that's my point. I cant wait to invoke io_service::run when i receive a packet, I need to handle that packet right away. That is why I need a different thread to always receive and handle the received packets right away. The send can be asynch but the receive has to be synch (or asynch and then calling right away the io_service::run which is basically the same).. Which means I need to have a different thread receiving packets and handling them all the time. And another thread handling some other stuff and sometimes sending... – grich Nov 02 '10 at 16:56
  • @grich no, you really don't understand how asynchronous operations work. Your application posts an asynchronous read, then invokes `io_service::run` exactly once. When the data arrives, your read handler is invoked. This is exactly the same behavior as a synchronous read, except the control flow is inverted. Inside the handler, you post additional async operations depending on what your application needs to do. You don't need a thread to 'always be receiving', that is what the `io_service` object does for you when using async operations. – Sam Miller Nov 02 '10 at 17:06
  • @Sam Miller OK I think i got you, But: According to boost, io_service::run is a blocking function which mean it will stop the thread that used it until all asynch actions are done.. So unless i got that wrong, I still need one thread running the blocking function io_service::run and another thread doing all the other work and sometimes posting a msg. Meaning I need the same number of threads as I needed in my first design, so why is it better? P.S I saw this example to understand: http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/example/chat/chat_client.cpp and they did another thread – grich Nov 02 '10 at 17:48
  • @grich `io_service::run` is a blocking function, the asynchronous handlers are invoked from inside of it. It only blocks as long as the `io_service` queue has remaining work to do. You only need a single thread. – Sam Miller Nov 02 '10 at 18:57
  • @Sam Miller yea but since in my receive handler function I'll always add an asynchronous receive, then it will always block... as the example I posted, in handle_read_header they added an async_read with handle_read_body, and in handle_read_body they added an async_read with handle_read_head.. Meaning it always add io_service::run work, so it always has remaining work to do.. Thats why they invoked io_service::run in a different thread. Did I finally get it or am i still wrong? tnx! – grich Nov 02 '10 at 22:20
  • @grich the in the async chat client example, the `io_service::run` method is invoked in a separate thread because the main thread performs a blocking read from `cin`. The same behavior could be achieved using a `posix::stream_descriptor` and invoking `io_service::run` from the main thread. – Sam Miller Nov 03 '10 at 14:03
  • @Sam Miller hmm actually io_service blocks the thread, I tried this code: http://pastebin.com/WcVmpnEQ only the `other_number` was increased and i always got `Main number is: 0 other number is: 1++(increasing each time)` and `Main_number` never increased... only when i tried running the run() function in a different thread, it worked and increased both numbers... so i dont understand how u suggest that i'll invoke `io_service::run` from the main thread maybe u can point out how to do it.. tnx a lot for your help! – grich Nov 03 '10 at 16:47
  • @grich you are using sleep, which is a blocking system call. I suggest you [read the tutorial](http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/tutorial/tuttimer2.html) about asio deadline timers. – Sam Miller Nov 03 '10 at 19:25
  • @Sam Miller I don't understand this... who runs the handles functions? the functions that u "call" using `io_service::post`? the thread that calls the io_service::run() or the thread of `io_service` (cuz i understood that it has its own thread from another answer here)? anyways in the code I linked (from pastbin) if i wouldn't do `Sleep` in there would `main_number` increase? cuz i tried that and it still doesn't increase if i don't do it in the same thread.. so I don't really understand how is what u wrote me in the last comment has connection to the topic.. can u go into details? tnx! – grich Nov 03 '10 at 22:18

3 Answers3

13

You need to decide first which style of socket communication you are going to use:

  1. synchronous - means that all low-level operations are blocking, and typically you need a thread for the accept, and then threads (read thread or io_service) to handle each client.

  2. asynchronous - means that all low-level operations are non-blocking, and here you only need a single thread (io_service), and you need to be able to handle callbacks when certain things happen (i.e. accepts, partial writes, result of reads etc.)

Advantage of approach 1 is that it's a lot simpler to code (??) than 2, however I find that 2 is most flexible, and in fact with 2, by default you have a single threaded application (internally the event callbacks are done in a separate thread to the main dispatching thread), downside of 2 of course is that your processing delay hits the next read/write operations... Of course you can make multi-threaded applications with approach 2, but not vice-versa (i.e. single threaded with 1) - hence the flexibility...

So, fundamentally, it all depends on the selection of style...

EDIT: updated for the new information, this is quite long, I can't be bothered to write the code, there is plenty in the boost docs, I'll simply describe what is happening for your benefit...

[main thread] - declare an instance of io_service - for each of the servers you are connecting to (I'm assuming that this information is available at start), create a class (say ServerConnection), and in this class, create a tcp::socket using the same io_service instance from above, and in the constructor itself, call async_connect, NOTE: this call is a scheduling a request for connect rather than the real connection operation (this doesn't happen till later) - once all the ServerConnection objects (and their respective async_connects queued up), call run() on the instance of io_service. Now the main thread is blocked dispatching events in the io_service queue.

[asio thread] io_service by default has a thread in which scheduled events are invoked, you don't control this thread, and to implement a "multi-threaded" program, you can increase the number of threads that the io_service uses, but for the moment stick with one, it will make your life simple...

asio will invoke methods in your ServerConnection class depending on which events are ready from the scheduled list. The first event you queued up (before calling run()) was async_connect, now asio will call you back when a connection is established to a server, typically, you will implement a handle_connect method which will get called (you pass the method in to the async_connect call). On handle_connect, all you have to do is schedule the next request - in this case, you want to read some data (potentially from this socket), so you call async_read_some and pass in a function to be notified when there is data. Once done, then the main asio dispatch thread will continue dispatching other events which are ready (this could be the other connect requests or even the async_read_some requests that you added).

Let's say you get called because there is some data on one of the server sockets, this is passed to you via your handler for async_read_some - you can then process this data, do as you need to, but and this is the most important bit - once done, schedule the next async_read_some, this way asio will deliver more data as it becomes available. VERY IMPORTANT NOTE: if you no longer schedule any requests (i.e. exit from the handler without queueing), then the io_service will run out of events to dispatch, and run() (which you called in the main thread) will end.

Now, as for writing, this is slightly trickier. If all your writes are done as part of the handling of data from a read call (i.e. in the asio thread), then you don't need to worry about locking (unless your io_service has multiple threads), else in your write method, append the data to a buffer, and schedule an async_write_some request (with a write_handler that will get called when the buffer is written, either partially or completely). When asio handles this request, it will invoke your handler once the data is written and you have the option of calling async_write_some again if there is more data left in the buffer or if none, you don't have to bother scheduling a write. At this point, I will mention one technique, consider double buffering - I'll leave it at that. If you have a completely different thread that is outside of the io_service and you want to write, you must call the io_service::post method and pass in a method to execute (in your ServerConnection class) along with the data, the io_service will then invoke this method when it can, and within that method, you can then buffer the data and optionally call async_write_some if a write is currently not in progress.

Now there is one VERY important thing that you must be careful about, you must NEVER schedule async_read_some or async_write_some if there is already one in progress, i.e. let's say you called async_read_some on a socket, until this event is invoked by asio, you must not schedule another async_read_some, else you'll have lots of crap in your buffers!

A good starting point is the asio chat server/client that you find in the boost docs, it shows how the async_xxx methods are used. And keep this in mind, all async_xxx calls return immediately (within some tens of microseconds), so there are no blocking operations, it all happens asynchronously. http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/example/chat/chat_client.cpp, is the example I was referring to.

Now if you find that performance of this mechanism is too slow and you want to have threading, all you need to do is increase the number of threads that are available to the main io_service and implement the appropriate locking in your read/write methods in ServerConnection and you're done.

Nim
  • 33,299
  • 2
  • 62
  • 101
  • yea ok what i dont get is this: i want to make a thread for always receiving and my "main thread" would sometimes send packets.. so I'll have to use the same socket for both operations, but the problem is i see it is not thread-safe to do so in this why.. so what should i do? – grich Nov 02 '10 at 15:29
  • Try not to think interms of threads, instead, update the above question with what it is you are intending to do, how do you want this server to work? May be easier to answer then... – Nim Nov 02 '10 at 15:33
  • I'm not doing a sever, I'm doing a client that connects to a-lot of servers. Anyways i edited my question to clarify everything better, Tnx. – grich Nov 02 '10 at 15:51
  • Tnx a lot but I didn't understand some parts of what you wrote. First thing first, some of my writing would be as a respond to the received data, and some of it wont be as a respond and just random writing. i didn't understand the double buffer thing you offered.. I want to write, so I call `io_service::post` with a method from `ServerConnection` (lets call it `sendMsg`).. and `sendMsg` should call `async_send` (why did u use `async_write_some` btw?) and that will make it a thread-safe operation? and before calling `async_send` i should lock and after it i should unlock? what about the recive? – grich Nov 03 '10 at 17:34
  • @Nim hi m8 can u read my last comment and please respond :)? Tnx – grich Nov 05 '10 at 10:16
  • @grich, day job is in the way! ;) From the external thread, you must call `io_service::post()`. Then the io_service will invoke the method when it's ready, at which point you can write the data to the buffer. Within the context of the io_service (let's say it's called you back on your read handler), you don't need to `post()`, you can simply append the data to a write buffer and call write. – Nim Nov 05 '10 at 10:52
  • btw, the method which is called via post must acquire a mutex so that it doesn't clash with a callback from io_service when a write is done (this is a problem if you have multiple threads in your io_service), if you have one, this is not a problem! What I meant by double buffering is that you should keep a pending buffer, to which you append new data (via the post() invoked method) or normal write request (in the context of the io_service), and swap this with an active buffer once the active buffer has been drained. In the handler that gets invoked after a partial write, you must check to see – Nim Nov 05 '10 at 10:55
  • ..if there is any more data in the active buffer, if so, write that first and then once that's done, swap that with the pending buffer. NOTE: none of these operations are thread safe if you have multiple threads in your io_service. – Nim Nov 05 '10 at 10:56
  • @Nim tnx for the help :) I do have multiple threads running `io_service::run`. What I understood from what you told me is that I should have one `async_read` max at all time and that i should have one `async_send` max at all time. I asked if I can have `async_send` and `async_recive` both at the same time and ppl said I can. For doing that, Sam Miller suggested using `boost:asio:strand` so I can make sure only 1 `async_write` is being called each time.. The others would have to wait, did I get it right? Is that what i meant when u said "e method which is called via post must acquire a mutex"? – grich Nov 05 '10 at 12:35
5

For asynchronous operations, you should use a single io_service object for the entire program. Whether its a static member of a class, or instantiated elsewhere is up to you. Multiple threads can invoke its run method, this is described in Inverse's answer.

Multiple threads may call io_service::run() to set up a pool of threads from which completion handlers may be invoked. This approach may also be used with io_service::post() to use a means to perform any computational tasks across a thread pool.

Note that all threads that have joined an io_service's pool are considered equivalent, and the io_service may distribute work across them in an arbitrary fashion.

if you have handlers that are not thread safe, read about strands.

A strand is defined as a strictly sequential invocation of event handlers (i.e. no concurrent invocation). Use of strands allows execution of code in a multithreaded program without the need for explicit locking (e.g. using mutexes).

Community
  • 1
  • 1
Sam Miller
  • 23,808
  • 4
  • 67
  • 87
  • 1
    I disagree with this answer. I've found out that multiple threads sharing an `io_service` can block each other. In my case, I had two threads, both adding a TCP read and a deadline timer, so the shared `io_service` object had 4 completion handlers to deal with. This caused one read to block on the other read, even though they did not share a socket. The problem was resolved by using per-thread `io_service` objects, each dealing with just one read and one timeout. – MSalters Oct 26 '16 at 16:32
1

The io_service is what invokes all the handler functions for you connections. So you should have one running for thread in order to distribute the work across threads. Here is a page explain the io_service and threads:

Threads and Boost.Asio

Inverse
  • 4,408
  • 2
  • 26
  • 35
  • wait so i need one for the "receiving thead" and one for the "sending thread"? or one for each socket? – grich Nov 02 '10 at 14:42
  • I see a discrepancy between your answer and the doc you linked to. The doc says that `run()` may be called by multiple threads to set up a thread pool. As I understand it's about multiple call of one instance's method and not about multiple instances. – user Feb 24 '14 at 14:04