0

I am having an issue with receive socket packet ordering using SocketAsyncEventArgs. The crux of my problem is that that when a client sends a packet to the server, the server will receive the packet in non-standard sized fragments and they will be processed in random order. This obviously means the packet cannot be decoded by my app which then screws up the entire conversation.

So for example, the client will send a full packet using the Socket.NetworkStream.Write() method:

 [-------PACKET-------]

The server, using SocketAsyncEventArgs, will get receive async callbacks on two separate packets but the last chunk of the packet will be processed first:

 First packet:  ET-------]
 Second packet: [-----PACK--

This does not happen on all packets and I haven't been able to accurately reproduce it based on packet size or timing. I implement a send/ack comms protocols so that the client won't send another packet until the server acks that the last packet was received successfully so it can't be that I'm overloading the server.

The most frustrating part is that Socket.Available on the server is always zero which according to the docs means nothing is available to read

 If you are using a non-blocking Socket, Available is a good way to determine whether data is
 queued for reading, before calling Receive. The available data is the total amount of data
 queued in the network buffer for reading. If no data is queued in the network buffer, 
 Available returns 0.

With available at zero, SocketEventArgs.Count doesn't seem to provide anything of value and the offset has to do with the receive buffer and not where it was based on the actual data stream I'm not sure how I can put these fragments in order.

My guess that the issue is the async callback for the first part of the packet gets preempted by the second callback which processes fully and then goes back to the first piece. The problem is I can't synclock the whole callback (wish .NET has a synchronized functions like Java). And even if I did, that seems like that would negate the benefits of async callback in the first place.

What am I doing wrong that makes these come in the wrong order or what can I do to make them be processed correctly?

Num Lock
  • 742
  • 1
  • 6
  • 32
  • TCP does not have packets or messages which you seem to assume. There are other false assumptions. Without code it is not possible to help you. Closing. – usr Mar 20 '15 at 13:41

1 Answers1

0

I am not quite sure what you are doing from some of your statements. You write you are using SocketAsyncEventArgs but are trying to deal with weird API stuff like .Count or .Available. If your socket type is TCP you are probably doing something really wrong, because packets will always be in the right order. They may be fragmented even to just one byte sized chunks, but the order will be right. That's pretty much what TCP is all about.

As you've provided no code and based on your statement, I guess it's best to just provide you with some SSCE to get you started.

The sample is in C# but should apply to VB.net. Check the comments in the code to see where to actually get the received data at from properly. The implementation will write the received data to the console and send it back to the client. Echo servers make great samples!

using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;

namespace SaeaSample
{
    public class Program
    {
        static void Main()
        {
            var server = new Server(new IPEndPoint(IPAddress.Any, 12345));

            // ugly sample clients
            Parallel.For(0, 4, i =>
            {
                using (var client = new TcpClient("localhost", 12345))
                using (var stream = client.GetStream())
                using (var writer = new BinaryWriter(stream))
                using (var reader = new BinaryReader(stream))
                {
                    var text = "Hello Async-Server!";
                    var message = Encoding.UTF8.GetBytes(text);
                    Console.WriteLine("s: {0}: {1}", i, text);
                    writer.Write(message);
                    var roundtrip = reader.ReadBytes(message.Length);
                    Console.WriteLine("r: {0}: {1}", i, Encoding.UTF8.GetString(roundtrip));
                }
            });

            Console.ReadLine();
        }
    }

    public class Server
    {
        private const int readBufferSize = 8192;
        private const int sendBufferSize = readBufferSize;

        // just have a fixed number of clients instead of
        // pooling for the sake of being an example
        private const int maxClients = 4;
        private const int maxQueue = 10;

        private readonly byte[] buffer = new byte[maxClients * (readBufferSize + sendBufferSize)];
        private readonly Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

        public Server(IPEndPoint localEndPoint)
        {
            socket.Bind(localEndPoint);
            socket.Listen(maxQueue);

            for (int i = 0; i < maxClients; i++)
            {
                var client = new UserToken(i);
                client.RecvArgs.Completed += completed;
                client.SendArgs.Completed += completed;

                Console.WriteLine("accepting on client slot {0}", client.Slot);

                if (!socket.AcceptAsync(client.RecvArgs))
                {
                    completed(this, client.RecvArgs);
                }
            }
        }

        private void completed(object sender, SocketAsyncEventArgs e)
        {
            var client = (UserToken)e.UserToken;

            // socket operation had success
            if (e.SocketError == SocketError.Success)
            {
                // new client connected
                if (e.LastOperation == SocketAsyncOperation.Accept)
                {
                    onAccept(client);
                }
                // either send or received worked
                else if (e.BytesTransferred > 0)
                {
                    if (e.LastOperation == SocketAsyncOperation.Receive)
                    {
                        onReceived(client);
                    }
                    else if (e.LastOperation == SocketAsyncOperation.Send)
                    {
                        onSend(client);
                    }
                    // should never happen, handle gracefully
                    else
                    {
                        onOther(client);
                    }
                }
                // don't handle anything else
                else
                {
                    onOther(client);
                }
            }
            // socket error occured
            else
            {
                onOther(client);
            }
        }

        private void onAccept(UserToken client)
        {
            Console.WriteLine("client slot {0} connected client from {1}", client.Slot, client.RecvArgs.AcceptSocket.RemoteEndPoint);

            // once accepted, start receiving
            client.RecvArgs.SetBuffer(buffer, client.Slot * (readBufferSize + sendBufferSize), readBufferSize);

            if (!client.RecvArgs.AcceptSocket.ReceiveAsync(client.RecvArgs))
            {
                completed(this, client.RecvArgs);
            }
        }

        private void onReceived(UserToken client)
        {
            // echo whatever we got
            var builder = new StringBuilder();

            // here is the important part
            for (int i = 0; i < client.RecvArgs.BytesTransferred; i++)
            {
                // offset the buffer and echo in hex
                builder.Append(client.RecvArgs.Buffer[client.Slot * (readBufferSize + sendBufferSize) + i].ToString("x2"));
            }
            Console.WriteLine("received {0} bytes from client slot {1}: {2}", client.RecvArgs.BytesTransferred, client.Slot, builder.ToString());

            // send data back ... this is an echo server after all
            client.SendArgs.SetBuffer(client.RecvArgs.Buffer, client.Slot * (readBufferSize + sendBufferSize) + readBufferSize, client.RecvArgs.BytesTransferred);
            Buffer.BlockCopy(client.RecvArgs.Buffer, client.RecvArgs.Offset, client.SendArgs.Buffer, client.SendArgs.Offset, client.RecvArgs.BytesTransferred);
            if (!client.RecvArgs.AcceptSocket.SendAsync(client.SendArgs))
            {
                completed(this, client.SendArgs);
            }
        }

        private void onSend(UserToken client)
        {
            Console.WriteLine("sent {0} bytes back to client slot {1}", client.SendArgs.BytesTransferred, client.Slot);

            // start receiving again
            if (!client.RecvArgs.AcceptSocket.ReceiveAsync(client.RecvArgs))
            {
                completed(this, client.RecvArgs);
            }
        }

        private void onOther(UserToken client)
        {
            Console.WriteLine("disconnecting client slot {0}", client.Slot);

            // just close the connection and accept again
            client.RecvArgs.SetBuffer(null, 0, 0);
            if (client.RecvArgs.AcceptSocket != null) {
                client.RecvArgs.AcceptSocket.Dispose();
                client.RecvArgs.AcceptSocket = null;
            }

            Console.WriteLine("accepting on client slot {0}", client.Slot);

            if (!socket.AcceptAsync(client.RecvArgs))
            {
                completed(this, client.RecvArgs);
            }
        }
    }

    public class UserToken
    {
        public readonly int Slot;
        public readonly SocketAsyncEventArgs RecvArgs = new SocketAsyncEventArgs();
        public readonly SocketAsyncEventArgs SendArgs = new SocketAsyncEventArgs();

        public UserToken(int slot)
        {
            Slot = slot;
            RecvArgs.UserToken = this;
            SendArgs.UserToken = this;
        }
    }
}

Also note that because this code is asynchronous the console outputs may or may not be always in order. You can decrease the read and write buffer size constants all the way down from 8192 to 1. The packets will be sent byte by byte in both directions, but will definitely be still in order.

For some more in depth explainations MSDN is always a good starting point.

Num Lock
  • 742
  • 1
  • 6
  • 32