13

First off, I hope my question makes sense and is even possible! From what I've read about TCP sockets and Boost::ASIO, I think it should be.

What I'm trying to do is to set up two machines and have a working bi-directional read/write link over TCP between them. Either party should be able to send some data to be used by the other party.

The first confusing part about TCP(/IP?) is that it requires this client/server model. However, reading shows that either side is capable of writing or reading, so I'm not yet completely discouraged. I don't mind establishing an arbitrary party as the client and the other as the server. In my application, that can be negotiated ahead of time and is not of concern to me.

Unfortunately, all of the examples I come across seem to focus on a client connecting to a server, and the server immediately sending some bit of data back. But I want the client to be able to write to the server also.

I envision some kind of loop wherein I call io_service.poll(). If the polling shows that the other party is waiting to send some data, it will call read() and accept that data. If there's nothing waiting in the queue, and it has data to send, then it will call write(). With both sides doing this, they should be able to both read and write to each other.

My concern is how to avoid situations in which both enter into some synchronous write() operation at the same time. They both have data to send, and then sit there waiting to send it on both sides. Does that problem just imply that I should only do asynchronous write() and read()? In that case, will things blow up if both sides of a connection try to write asynchronously at the same time?

I'm hoping somebody can ideally:

1) Provide a very high-level structure or best practice approach which could accomplish this task from both client and server perspectives

or, somewhat less ideally,

2) Say that what I'm trying to do is impossible and perhaps suggest a workaround of some kind.

Sam Miller
  • 23,808
  • 4
  • 67
  • 87
aardvarkk
  • 14,955
  • 7
  • 67
  • 96

4 Answers4

3

What you want to do is absolutely possible. Web traffic is a good example of a situation where the "client" sends something long before the server does. I think you're getting tripped up by the words "client" and "server".

What those words really describe is the method of connection establishment. In the case of "client", it's "active" establishment; in the case of "server" it's "passive". Thus, you may find it less confusing to use the terms "active" and "passive", or at least think about them that way.

With respect to finding example code that you can use as a basis for your work, I'd strongly encourage you to take a look at W. Richard Stevens' "Unix Network Programming" book. Any edition will suffice, though the 2nd Edition will be more up to date. It will be only C, but that's okay, because the socket API is C only. boost::asio is nice, but it sounds like you might benefit from seeing some of the nuts and bolts under the hood.

Chris Cleeland
  • 4,760
  • 3
  • 26
  • 28
2

My concern is how to avoid situations in which both enter into some synchronous write() operation at the same time. They both have data to send, and then sit there waiting to send it on both sides. Does that problem just imply that I should only do asynchronous write() and read()? In that case, will things blow up if both sides of a connection try to write asynchronously at the same time?

It sounds like you are somewhat confused about how protocols are used. TCP only provides a reliable stream of bytes, nothing more. On top of that applications speak a protocol so they know when and how much data to read and write. Both the client and the server writing data concurrently can lead to a deadlock if neither side is reading the data. One way to solve that behavior is to use a deadline_timer to cancel the asynchronous write operation if it has not completed in a certain amount of time.

You should be using asynchronous methods when writing a server. Synchronous methods are appropriate for some trivial client applications.

Sam Miller
  • 23,808
  • 4
  • 67
  • 87
  • This is getting very close to what I'm looking for. Where can I read about or see example of these higher-level application protocols for simple bi-directional communication? – aardvarkk Jun 24 '11 at 01:32
  • @aardvarkk protocols are largely application dependent, I suggest asking another question with more specific information about your application and how to design its protocol. – Sam Miller Jun 24 '11 at 03:30
1

TCP is full-duplex, meaning you can send and receive data in the order you want. To prevent a deadlock in your own protocol (the high-level behaviour of your program), when you have the opportunity to both send and receive, you should receive as a priority. With epoll in level-triggered mode that looks like: epoll for send and receive, if you can receive do so, otherwise if you can send and have something to send do so. I don't know how boost::asio or threads fit here; you do need some measure of control on how sends and receives are interleaved.

Tobu
  • 24,771
  • 4
  • 91
  • 98
  • @tc. damn, you're right. letting the send buffers fill up in a non-blocking way would force the program to switch to receiving. I'll just catch some sleep. – Tobu Jun 24 '11 at 00:01
  • 1
    So the fact that TCP is full-duplex would imply that it's no big deal for both sides to try writing at the same time -- I just have to make sure I don't wait indefinitely for writes to complete because then I'd encounter a deadlock situation. Is that correct? – aardvarkk Jun 24 '11 at 01:29
0

The word you're looking for is "non-blocking", which is entirely different from POSIX asynchronous I/O (which involves signals).

The idea is that you use something like fcntl(fd,F_SETFL,O_NONBLOCK). write() will return the number of bytes successfully written (if positive) and both read() and write() return -1 and set errno = EAGAIN if "no progress can be made" (no data to read or write window full).

You then use something like select/epoll/kqueue which blocks until a socket is readable/writable (depending on the flags set).

tc.
  • 33,468
  • 5
  • 78
  • 96