C# Asynchronous Server Sockets - Thread-Safety/Performance (MMO Gaming)

Question

I'm writing this game server for my small 2D MMO game.

So here are my questions:

1. What do you think about the Thread-Safety of the code? can you show me where the problems are and how to fix them / patch them?

Here is my code:

using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;

public class Constanti
{
    public const int CLASS_DARKELF_MAGICIAN = 1;
    public const int CLASS_HUMAN_MAGICIAN   = 2;
    public const int CLASS_WARRIOR          = 3;
    public const int CLASS_MODERN_GUNMAN    = 4;
    public const int SUIT_1 = 1;
    public const int SUIT_2 = 2;
    public const int SUIT_3 = 3;
    public const int SUIT_4 = 4;
    public const int SUIT_Admin = 5;

    //MAX/MIN
    public const int MAX_LEVEL = 100;
    public const int MAX_SKILL_LEVEL = 1000;

    //SERVER MAX/MIN
    public const int MAX_CONNECTIONS = 300;
    public const int MAX_CONNECTIONS_IP = 4;
}

// State object for reading client data asynchronously
public class Player
{
    // Client  socket.
    public Socket workSocket = null;
    // Size of receive buffer.
    public const int BufferSize = 1024;
    // Receive buffer.
    public byte[] buffer = new byte[BufferSize];
    // Received data string.
    public StringBuilder sb = new StringBuilder();
    //Player-Info
    public int PlayerStats_Health = 0;
    public int PlayerStats_Energy = 0;
    public int PlayerInfo_Class = 0;
    public int PlayerInfo_Suit = 0;
    public int PlayerInfo_Level = 0;
    public int PlayerInfo_SkillLevel = 0;

    public void SetDefaults()
    {
        PlayerStats_Health = 100;
        PlayerStats_Energy  = 200;
        PlayerInfo_Class = Constanti.CLASS_DARKELF_MAGICIAN;
        PlayerInfo_Suit = Constanti.SUIT_1;
        PlayerInfo_Level = 1;
        PlayerInfo_SkillLevel = 1;
    }

    public Player()
    {
    }

    public String pIPAddress;
}

public class GameObjectLists
{
    public static List<Player> PlayersList = new List<Player>();
}

public class AsynchronousSocketListener
{
    // Thread signal.
    public static ManualResetEvent allDone = new ManualResetEvent(false);
    public static int PlayersOnline = 0;
    public AsynchronousSocketListener()
    {}

    public static void InitializeMySQL()
    {
        //TODO MySQLI/MySQL Connection
    }

    public static void MysqlUpdateQuery()
    {
        //Mysql UPDATE, no return stmt
    }

    public static String MySQLSelect()
    {
        //TODO MySQL Select
        String retdata="test";

        return retdata;
    }

    public static void StartListening()
    {
        // Data buffer for incoming data.
        byte[] bytes = new Byte[1024];
        // Establish the local endpoint for the socket.
        // The DNS name of the computer
        /*
        IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
        IPAddress ipAddress = ipHostInfo.AddressList[0];*/

        IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Any, 86);

        // Create a TCP/IP socket.
        Socket listener = new Socket(AddressFamily.InterNetwork,SocketType.Stream, ProtocolType.Tcp);

        // Bind the socket to the local endpoint and listen for incoming connections.
        try
        {
            listener.Bind(localEndPoint);
            listener.Listen(50);
            Console.WriteLine("Server Started, waiting for connections...");

            while (true)
            {
                // Set the event to nonsignaled state.
                allDone.Reset();

                // Start an asynchronous socket to listen for connections.

                listener.BeginAccept(
                    new AsyncCallback(AcceptCallback),
                    listener);

                // Wait until a connection is made before continuing.
                allDone.WaitOne();
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e.ToString());
        }

        //Console.WriteLine("\nPress ENTER to continue...");
        Console.Read();
    }


    public static void AcceptCallback(IAsyncResult ar)
    {
        // Get the socket that handles the client request.
        Socket listener     = (Socket)ar.AsyncState;
        Socket clientsocket = listener.EndAccept(ar);
        // Signal the main thread to continue.
        allDone.Set();
        clientsocket.Blocking = false;      //set to non-blocking
        // Create the state object.
        Player PlayerInfo = new Player();
        PlayerInfo.workSocket = clientsocket;

        IPEndPoint thisIpEndPoint = PlayerInfo.workSocket.RemoteEndPoint as IPEndPoint; //Get Local Ip Address
        PlayerInfo.pIPAddress = thisIpEndPoint.Address.ToString();

        GameObjectLists.PlayersList.Add(PlayerInfo);
        PlayersOnline++;

        int numconnsofip = 0;
        GameObjectLists.PlayersList.ForEach(delegate(Player PlayerInfoCheck)
        {
                //Console.WriteLine(name);
                if (PlayerInfoCheck.pIPAddress == PlayerInfo.pIPAddress)
                {
                    numconnsofip++;
                }
        });

        if (PlayersOnline > Constanti.MAX_CONNECTIONS || numconnsofip > Constanti.MAX_CONNECTIONS_IP)
        {
            Disconnect(clientsocket, PlayerInfo);
        }
        else
        {
            Console.WriteLine("Player with IP:[{0}] has [{1}] Connections", thisIpEndPoint.Address.ToString(), numconnsofip);
            PlayerInfo.SetDefaults();
            //clientsocket.LingerState = new LingerOption(true, 2);    // give it up to 2 seconds for send
            Console.WriteLine("New Connection Total:[{0}]", PlayersOnline);
            clientsocket.BeginReceive(PlayerInfo.buffer, 0, Player.BufferSize, 0, new AsyncCallback(ReadCallback),
                PlayerInfo);
        }
    }

    public static void ProtocolCore(Player PlayerInfo, String data)
    {
        Console.WriteLine("Procesing Packet:{0}",data);
        //if data == bla bla then send something to everyone:

        GameObjectLists.PlayersList.ForEach(delegate(Player ObjPlayerInfo)
        {
            Send(data,ObjPlayerInfo);
        });
    }

    public static void ReadCallback(IAsyncResult ar)
    {
        // TEST #1 - IF WE HANG HERE, THERE WILL BE STILL OTHER CONNECTIONS COMING HERE, BUT NO MULTI THREADING?? 
        // Retrieve the state object and the clientsocket socket
        // from the asynchronous state object.
        Player PlayerInfo = (Player)ar.AsyncState;
        Socket clientsocket = PlayerInfo.workSocket;
        try
        {
            String content = String.Empty;  //content buffer

            // Read data from the client socket. 
            // IF THIS FAILS, WE CATCH / ASSUMING THAT:
            // THE CLIENT FORCE-CLOSED THE CONNECTION OR OTHER REASON.
            int bytesRead = clientsocket.EndReceive(ar);    

            if (bytesRead > 0)
            {
                // There  might be more data, so store the data received so far.

                PlayerInfo.sb.Append(Encoding.ASCII.GetString(
                    PlayerInfo.buffer, 0, bytesRead));

                // Check for end-of-file tag. If it is not there, read 
                // more data.
                content = PlayerInfo.sb.ToString();
                int eofindex = content.IndexOf("<EOF>");
                if (eofindex > -1)
                {
                    // All the data has been read from the 
                    // client. Display it on the console.
                    content = content.Substring(0,eofindex);  //remove THE <EOF>

                    Console.WriteLine("Read {0} bytes from socket. Data : {1}",content.Length, content);

                    //PROCESS THE PACKET/DATA (PROTOCOL CORE)
                    ProtocolCore(PlayerInfo, content);

                    //Echo the data back to the client.
                    Send(content, PlayerInfo);
                    // CLEAR THE BUFFERS
                    PlayerInfo.sb.Remove(0, PlayerInfo.sb.Length);
                    Array.Clear(PlayerInfo.buffer, 0, PlayerInfo.buffer.Length);

                    // GO TO LISTEN FOR NEW DATA
                    clientsocket.BeginReceive(PlayerInfo.buffer, 0, Player.BufferSize, 0,
                    new AsyncCallback(ReadCallback), PlayerInfo);
                }
                else
                {
                    // Not all data received. Get more.
                    clientsocket.BeginReceive(PlayerInfo.buffer, 0, Player.BufferSize, 0,
                    new AsyncCallback(ReadCallback), PlayerInfo);
                }
            }
            else
            {
                //ASSUMING WE RECEIVED 0 SIZED PACKET or CLIENT DISCONNECT / THEREFORE CLOSE THE CONNECTION
                Disconnect(clientsocket, PlayerInfo);
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e.ToString());
            Disconnect(clientsocket, PlayerInfo);
        }
    }

    private static void Send(String data,Player PlayerInfo)
    {
        // Convert the string data to byte data using ASCII encoding.
        byte[] byteData = Encoding.ASCII.GetBytes(data);

        // Begin sending the data to the remote device.
        PlayerInfo.workSocket.BeginSend(byteData, 0, byteData.Length, 0,
            new AsyncCallback(SendCallback), PlayerInfo);
    }

    private static void Disconnect(Socket clientsocket, Player PlayerInfo)
    {

        try
        {
            PlayersOnline--; //Is this Thread-Safe also?
            GameObjectLists.PlayersList.Remove(PlayerInfo);
            Console.WriteLine("Socket Disconnected, PlayerObjects:[{0}]", GameObjectLists.PlayersList.Count);
            clientsocket.Shutdown(SocketShutdown.Both);
            clientsocket.Close();
        }
        catch (Exception e)
        {
           Console.WriteLine(e.ToString());
        }
    }

    private static void SendCallback(IAsyncResult ar)
    {
        // Retrieve the socket from the state object.
        Player PlayerInfo = (Player)ar.AsyncState;
        Socket clientsocket = PlayerInfo.workSocket;
        try
        {
            // Complete sending the data to the remote device.
            int bytesSent = clientsocket.EndSend(ar);
            Console.WriteLine("Sent {0} bytes to client.", bytesSent);
        }
        catch (Exception e)
        {
            Console.WriteLine(e.ToString());
            Disconnect(clientsocket, PlayerInfo);
        }
    }


    public static int Main(String[] args)
    {
        InitializeMySQL();
        StartListening();
        return 0;
    }
}

Answer 1

The first thing I want to mention, since I believe it is answers the root of your question, is that your performance (latency, concurrent connection capacity, etc) is going to be largely defined by the hardware you are running this software on and the network performance for each specific client. Software can improve some things but generally speaking, if your code is well written and understandable by others and contains no bugs, it will perform just fine.

Will your code handle 300 connections concurrently? Yes most likely it can. I do see some potential issues with threading though. One being that you will have a lot of contention when you accept new clients. You also created a vulnerability by waiting between accepts for each client to be fully accepted, a potential for a denial of service attack. A client can stall the connection by requiring re-transmissions of data for each packet up to three times per packet and can wait to deliver each message up to whenever your timeout is (10 seconds?). There will also be a lot of problems with data processing unless the method stubs you have are thread safe themselves once you implement them.

You are using the older asynchronous socket model. It's a little complicated for what it is. I think that you will understand the event driven model a little better since it is more natural in my opinion. I know that from my experience, either perform just fine. However, I have also found that the new event driven model is a bit faster since you do not have huge garbage collections being done due to over allocation of IAsyncResult objects. The newer model uses methods such as Socket.AcceptAsync and the Socket.Completed event.

Since you are new to C#, I would recommend that instead you should be focusing efforts on writing a simple and clean client/server application that has asynchronous elements. You can do a load test on that and see if it satisfies your criteria for performance in terms of the raw throughput on your hardware. This will reduce the factors in your analysis. I recommend starting with something that can communicate simple text messages back and forth. You can increase the complexity as you gain a deeper understanding of the nuances of threading and network communication in .NET.

One website I recommend you looking into is http://www.albahari.com/threading/, this has a good explanation of the various building blocks for writing multi-threaded code. It's aimed at people with programming experience, as you mentioned you have. The "advanced threading" section is what I reference often.

The book "Pro C# 2010 and the .NET 4 Platform" by Andrew Troelson (ISBN 1430225491) would be a good start as well. It covers a lot of the language and shows some parallels between C# and other languages. It also covers a lot of .NET which is what most people really need to get a good understanding of before diving into the fun stuff.