1

I'm trying to connect to a remote machine using the TcpClient class, but it keeps failing:

An unhandled exception of type 'System.Net.Sockets.SocketException' occurred in System.dll

Additional information: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

Testing the code when the client and server are local works, but when I try connecting to a remote machine, it no longer works.

Here is the server code:

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Net.WebSockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace WebSocketServer
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Starting a new WebSockets server.");
            WebSocketServer server = new WebSocketServer();
            Console.WriteLine("The WebSocket server has started.");
            bool userRequestedShutdown = false;
            while (!userRequestedShutdown)
            {
                Console.ReadLine();
                DialogResult result = MessageBox.Show("Do you want to shut the server down?", "Warning", MessageBoxButtons.YesNo, MessageBoxIcon.Warning);
                if (result == DialogResult.Yes)
                {
                    userRequestedShutdown = true;
                }
            }
            server.Stop();
        }

        class WebSocketServer
        {

            TcpListener server;
            Thread connectionListener;
            ConcurrentDictionary<TcpClient, Thread> clients = new ConcurrentDictionary<TcpClient, Thread>();

            public WebSocketServer()
            {
                server = new TcpListener(IPAddress.Parse("127.0.0.1"), (int)Properties.Settings.Default["Port"]);
                try
                {
                    server.Start();
                }
                catch (Exception exception)
                {
                    Console.WriteLine("Error while trying to start the server: {0}", exception.ToString());
                }
                connectionListener = new Thread(() =>
                {
                    while (true)
                    {
                        Console.WriteLine("Waiting for a new client.");
                        try
                        {
                            TcpClient client = server.AcceptTcpClient();
                            Thread clientListener = new Thread(() =>
                            {
                                try
                                {
                                    NetworkStream stream = client.GetStream();
                                    byte[] buffer = new byte[1024];
                                    Console.WriteLine("Wating for the client to write.");
                                    while (client.Connected)
                                    {
                                        try
                                        {
                                            int bytesRead = stream.Read(buffer, 0, buffer.Length);
                                            Console.WriteLine("Read {0} bytes from the client.", bytesRead);
                                            string data = Encoding.UTF8.GetString(buffer).Substring(0, bytesRead);
                                            Console.WriteLine("Read the following string from the client: {0}", data);
                                        }
                                        catch (Exception exception)
                                        {
                                            Console.WriteLine("Error while trying to read from a TCP client: {0}", exception.ToString());
                                            break;
                                        }
                                    }
                                    Console.WriteLine("Client disconnected. Removing client.");
                                }
                                catch (Exception exception)
                                {
                                    Console.WriteLine("Error while trying to connect to a TCP client: {0}", exception.ToString());
                                }
                                client.Close();
                                clients.TryRemove(client, out clientListener);
                            });
                            clientListener.Start();
                            clients.TryAdd(client, clientListener);
                        }
                        catch (Exception exception)
                        {
                            Console.WriteLine("Error while trying to accept a TCP client: {0}", exception.ToString());
                        }
                        Console.WriteLine("A client has connected.");
                    }
                });
                connectionListener.Start();
            }

            public void Stop()
            {
                server.Stop();
                connectionListener.Abort();
            }
        }
    }
}

Here is the client code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace WebSocketClient
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Opening up a TcpClient.");
            TcpClient client = new TcpClient();
            client.Connect("<Remote Hostname>", <RemotePortNumber>);
            Console.WriteLine("TcpClient has connected.");
            NetworkStream stream = client.GetStream();
            bool closed = false;
            new Thread(() =>
            {
                while (!closed)
                {
                    Console.WriteLine("Writing data to the stream.");
                    byte[] bytes = Encoding.UTF8.GetBytes("Hello, world.");
                    stream.Write(bytes, 0, bytes.Length);
                    Thread.Sleep(1000);
                }
            }).Start();
            Console.ReadLine();
            closed = true;
        }
    }
}

So what is the problem here? I am hosting the server on an Azure Virtual Machine, I have opened up the TCP port I am trying to use as the <RemotePortNumber> in Windows Firewall on my remote server by setting both inbound and outbound rules to allow all traffic in and out of the machine on that port, and I have created a TCP endpoint on my Azure portal that maps the external port of my Virtual Machine's hostname to the internal, private port, of my Virtual Machine, both set to map the same port number of <RemotePortNumber> for consistency. To connect, I am using a <Remote Hostname> value of <MyServiceName>.cloudapp.net. I have also tried connecting the stream by using IPAddress.Parse(<Public IP of my Azure Server>) but have had no luck...it just keeps timing out, as if I am not formatting the hostname correctly or something. What am I missing? If anyone can provide some clues as to how to debug the issue, that would also be very helpful.

Update: Running a WireShark trace, I see a lot of these messages (is this bad? I think the TCP Retransmission might be okay if we take into account that for Azure you have to route packets from the public domain's port to the private port of the VM, but not sure about whatever the RST, ACK's are):

enter image description here

Update: Running Microsoft Message Analyzer, I see these messages on the Local Link Layer:

enter image description here

Note: My VM has an Internal IP of 100.75.20.78 and a Public IP of 191.238.37.130. It has the public domain name of ovidius.cloudapp.net. I am trying to host the application on TCP port 6490. I blacked out my personal IP address for the sake of not giving it up.

I have mapped the TCP port in the Azure portal from domain to VM as follows:

enter image description here

Alexandru
  • 12,264
  • 17
  • 113
  • 208
  • 1
    A hint, try to telnet to your server. If it succeeds, the problem is very likely to be at the client. If not, the problem is at the server. – Vinícius Gobbo A. de Oliveira Jun 15 '14 at 00:49
  • @alex You can also try traceroute to see how far you're making it to the server. Can you tcpdump on the server to verify that responses are being generated back to the client? – CDahn Jun 15 '14 at 00:55
  • @ViníciusGobboA.deOliveira Just tried to telnet open . I got, "Could not open connection to the host, on port"...its probably the server then...I also tried to tracert my but it didn't want to go through...kept getting "Request timed out." – Alexandru Jun 15 '14 at 00:59
  • @CDahn Even tracert for my server's public IP seems to be timing out. Weird or what? Is there a way to specify a port when using tracert? I have only ports 5986, 63225, 1337, and 1338 open on the server. Also, how do you run a tcpdump? Is that part of some Windows feature? – Alexandru Jun 15 '14 at 01:09
  • The `SocketException` type has some nice properties like `ErrorCode` and --if I remember right-- `SocketErrorCode` or `SocketError` which is a nice readable enum that you can inspect. Those will tell you exactly what the problem is and, usually, exactly where the problem lies. – clarkitect Jun 15 '14 at 01:15
  • @jeffdot The error code is 10060: WSAETIMEDOUT – Alexandru Jun 15 '14 at 01:19
  • Side note: if you need web sockets you may consider using IIS/ASP.Net for that (assuming your code is not tied to older versions of Windows). – Alexei Levenkov Jun 15 '14 at 01:40
  • Guys, I installed WireShark to try and capture the TCP layer and I'm noticing a TCP Retransmission come up in red when I try to connect the client to the server. Does this indicate that packets are getting lost, and if so, does anyone have any ideas as to how to troubleshoot that? – Alexandru Jun 15 '14 at 01:50
  • @AlexeiLevenkov I don't think that using IIS would shoot this problem down. – Alexandru Jun 15 '14 at 01:52
  • It may make configuration easier as regular web sites are pretty common for Azure and done by more people than low level TCP servers. You'll likely find more guides/help/built in configuration support if you IIS. As an option seek for other guides for installing other Web Socket servers (i.e. NodeJS one) on Azure. – Alexei Levenkov Jun 15 '14 at 01:56
  • @AlexeiLevenkov I want to program a back-end web service using a low-level TCP server like this for a cross-platform multiplayer game because its well supported natively across all platforms. I'll try to Google some Azure install steps for other back-end servers and see if anything turns up in my case for this type of a server. – Alexandru Jun 15 '14 at 02:06
  • @AlexeiLevenkov Its way too specific to node.js: http://azure.microsoft.com/en-us/documentation/articles/cloud-services-nodejs-develop-deploy-app/...last time I opened up a TCP port successfully, I ran it through a local Windows Service that hosted a WCF endpoint. I wonder if, since I'm running the server using Administrator credentials, its a permissions thing. – Alexandru Jun 15 '14 at 02:10
  • @AlexeiLevenkov Just tried to do the same thing I did with a previous WCF service over NET.TCP by running the socket server code above under system credentials hosted in a Local Service but still no luck, so I know the issue isn't because of the account; its definitely these sockets for some reason...I've also tried rebooting the VM, on the off-chance that it might be the issue. Nothing. I still see the annoying TCP Retransmission from a DNS called menandmice-dns...so I guess I know what IP Address Management the Azure Cloud uses: http://www.menandmice.com/...oh man, this is such a nightmare. – Alexandru Jun 15 '14 at 02:27
  • @AlexeiLevenkov WCF services seem to work fine but I can't use them because they use the WC foundation on top of the TCP layer, so I need basic TCP...because of my cross-platform constraint. – Alexandru Jun 15 '14 at 02:28
  • I made a mistake in saying that's what Microsoft uses, its Wireshark telling me that's a general port (1337) for that particular product...well, I've been experimenting with different ports and settings on Azure to get it up and running, I will post back if I find anything but so far nothing... – Alexandru Jun 15 '14 at 02:58
  • I switched ports to 6490 as nothing should be assigned to those according to the IETF. I also tried to enable the Direct Server Return feature on the endpoint mapping of the domain's port of 6490 to the VM's port of 6490. Direct server return configures a virtual machine's endpoint for the floating IP capability required to configure a SQL AlwaysOn Availability Group...was worth a shot I guess, anyways...I tried connecting the client again, and I took a picture of the WireShark traces. Question has been updated! Please let me know if you guys spot anything that's off! – Alexandru Jun 15 '14 at 03:10
  • @Alex the RST packets are emanating from a different port (80) than what you're sending to. They could be unassociated, especially since there's more of them than your requests. The retransmissions are because the server isn't responding. – CDahn Jun 15 '14 at 07:45
  • @CDahn But this is so strange, because the application hasn't thrown any errors, I mean...the server seems to be up and listening on that port, yet packets are not reaching it for some reason, and I have configured both inbound and outbound rules on that port to allow all traffic from Windows Firewall. Yet, packets are still making it to the virtual machine...? – Alexandru Jun 15 '14 at 15:39
  • This could be a duplicate of http://stackoverflow.com/questions/19067869/why-are-my-new-azure-end-points-not-working – CDahn Jun 15 '14 at 19:04
  • @CDahn In that question, the user has Ubuntu running on their server. Also, I have tested that the endpoints I have open do in fact work because I can host a WCF NET.TCP binding on the port in question, however, I can't get a Socket TCP port up and running as per the code above. I did try Microsoft Message Analyzer. I will post the results in the question above...it may not be too helpful but I'll post all the IP's in question. – Alexandru Jun 15 '14 at 20:10
  • Updated the question again...I'm starting to wonder if mapping a particular domain port to the same internal port is an issue... – Alexandru Jun 15 '14 at 20:30
  • Solved. Solution posted. Comes down to 2 issues: 1. ENABLE DIRECT SERVER RETURN being enabled on the endpoint is a problem. 2. The server needs to be hosted on the Internal IP of the VM, not localhost or 127.0.0.1... – Alexandru Jun 15 '14 at 21:22

1 Answers1

3

After spending two mother _______ days on this (fill in the blanks), I got a little bit creative in ways of exploring alternative methods to try and see if I could rule routing issues out. I can pretty much safely conclude that Microsoft's Virtual Machine routing fails when you map a Public Port to the same Private Port.

For example, I tried setting up the new Socket endpoint below, and it worked because it didn't map the same domain port to the same Virtual Machine port like I had previously done with the WebSocketServer:

enter image description here

Update: Also, when hosting, I had to set up the server, not on the IP of 127.0.0.1, but the Internal IP, which in my case is 100.75.20.78.

Update Again: Contrary to the above solution, I tried to delete the old endpoint at 6490 and recreated it, and it seems to be working when I connect to that address now. I'm not entirely sure why, I can only say that the only difference here is that I had firewall rules on to allow that endpoint's port before creating the endpoint this time...not sure if that would make a difference. I'm honestly not sure what was causing the issues.

Update Yet Again: Just thought about it some more...I think it comes down to the following two issues:

  1. You need to host the server on the Internal IP of your Azure Virtual Machine, not localhost or 127.0.0.1 like I was doing.
  2. You need to not have the "ENABLE DIRECT SERVER RETURN" feature enabled on your Azure endpoint.
Alexandru
  • 12,264
  • 17
  • 113
  • 208