HTTP GET (with Header 'Connection: keep alive') over TCP

Question

I want to do the following with a raw C# socket. I understand that usually the most appropriate way is via HTTP, with a HTTP client. The browser understands that this connection must be kept open in some way.

http://server.domain.com/protocol/do_something.txt

I am trying the following in C#, but have had no luck. What am I doing wrong? Is there a header missing? Should I be encoding what I'm sending to the server in some way? For the ReceiverSocket client, I'm using the following code, but it's just a very standard asynchronous socket client: https://stackoverflow.com/a/10390066/971580

ReceiverSocket socket = new ReceiverSocket("server.domain.com", 80);
socket.Connect();
System.Threading.Thread.Sleep(1000);            
String message = "GET /protocol/do_something.txt HTTP/1.1";
message += "\r\n";
message += "\r\n";
socket.Send(message);

The socket can connect successfully, but I don't get any response when I send anything to the server. This is how I am connecting, sending and receiving.t (Apologies: I tried to do this in snippets, rather than including all the methods, but it looked horrid. . .)

    public ReceiverSocket(String address, int port) : base(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
    {
        messageQueue = new Queue<MessageBase>();
        IPHostEntry ipHostInfo = Dns.GetHostEntry(address);
        IPAddress ipAddress = ipHostInfo.AddressList[0];
        remoteEP = new IPEndPoint(ipAddress, port);
    }

    public void Connect()
    {
        this.BeginConnect(remoteEP, ConnectCallback, this);
    }

    private void ConnectCallback(IAsyncResult ar)
    {
        try
        {
            Socket client = (Socket)ar.AsyncState;
            if (client.Connected)
            {
                client.EndConnect(ar);
                Console.WriteLine("Connect Callback - Connected");
                StateObject state = new StateObject();
                state.workSocket = client;
                state.BufferSize = 8192;
                if (SocketConnected != null)
                    SocketConnected(client);

                client.BeginReceive(state.Buffer, state.readOffset, state.BufferSize - state.readOffset, 0, ReceiveCallback, state);
            }
            else
            {
                Thread.Sleep(5000);
                Connect();
            }
        }
        catch (Exception ex)
        {
            Reconnect();
        }
    }

    private void ReceiveCallback(IAsyncResult ar)
    {
        Console.WriteLine("Never gets here. . . ");
        try
        {
            StateObject state = (StateObject)ar.AsyncState;
            Socket client = state.workSocket;
            if (client.Connected)
            {
                int bytesRead = client.EndReceive(ar);
                foreach (MessageBase msg in MessageBase.Receive(client, bytesRead, state))
                {
                    // Add objects to the message queue
                    lock (this.messageQueue)
                        this.messageQueue.Enqueue(msg);
                }


                if (DataRecieved != null)
                    DataRecieved(client, null);

                client.BeginReceive(state.Buffer, state.readOffset, state.BufferSize - state.readOffset, 0, ReceiveCallback, state);
            }
            else
            {
                Reconnect();
            }
        }
        catch (SocketException)
        {
            Reconnect();
        }
    }

    public void Send(String msg)
    {
        try
        {
            byte[] bytes = GetBytes(msg);

            if (this.Connected)
            {

                Console.WriteLine("Sending: " + msg);
                this.BeginSend(bytes, 0, bytes.Length, 0, SendCallback, this);
            }
            else
            {
                Reconnect();
            }
        }
        catch (SocketException sox)
        {
            Reconnect();
        }
        catch (Exception ex)
        {
            int i = 0;
        }
    }


    static byte[] GetBytes(string str)
    {
        byte[] bytes = new byte[str.Length * sizeof(char)];
        System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
        return bytes;
    }
    }

    public class StateObject
    {
        public Socket workSocket = null;
        public int readOffset = 0;
        public StringBuilder sb = new StringBuilder();

        private int bufferSize = 0;
        public int BufferSize
        {
            set
            {
                this.bufferSize = value;
                buffer = new byte[this.bufferSize];
            }

            get { return this.bufferSize; }
        }

        private byte[] buffer = null;
        public byte[] Buffer
        {
            get { return this.buffer; }
        }
    }

Shouldn't the fact that I haven't included the message += "Connection: close" header mean that the socket should just start sending whatever data it has asynchronously? Just to note also: I can connect successfuly using Telnet and send the data, just not with a socket yet!

Any pointers at all would be appreciated.

Thanks.

Don't send `\r\n`, just send `\n`. Per the HTTP 1.1 RFC, you also need to send a `Host` header: _A client MUST include a Host header field in all HTTP/1.1 request messages_ — Sean Bright, Sep 14 '12 at 11:04
If there is no reply after sending `"\r\n\r\n"`, then there may be something wrong with that `ReceiverSocket` class that prevented it from truly sending the message. — E_net4, Sep 14 '12 at 11:08
@SeanBright The RFC2616 that you mentioned also states the use of `\r\n` as the end-of-line marker for all protocol elements except the entity-body. And even though the `Host` header isn't present, that would still make the server reply with `400 Bad Request`. — E_net4, Sep 14 '12 at 11:14
@TarbhJohnnyDick, so, why not use proper abstraction? I am out of curiosity, what is the reason? — Sergey Vyacheslavovich Brunov, Sep 14 '12 at 11:52
Query strings should not impede the server from replying to an HTTP request. I advise you to recheck `ReceiverSocket`. Can we also know what you are using to receive the response? — E_net4, Sep 14 '12 at 11:54
When @Serge says `WebClient`, he means `System.Net.WebClient`, which unless you have specific reason not to, I would also recommend you use. And nothing you've written so far says to me that you have any such reason. http://msdn.microsoft.com/en-us/library/system.net.webclient.aspx — tomfanning, Sep 14 '12 at 13:17
`System.Net.WebClient` does this: http://stackoverflow.com/a/4699045/17971 — tomfanning, Sep 14 '12 at 13:48
Then fire up Wireshark and execute that code to observe that WebClient uses HTTP Keep-Alive. — tomfanning, Sep 14 '12 at 13:51

score 0 · Answer 1 · answered Sep 16 '12 at 16:06

Do NOT call Send() until ConnectCallback() is called first, otherwise you risk sending your data prematurely. Using Sleep() to wait for the connection is wrong.

Do NOT call BeginReceive() until after Send() has finished sending the data.

Because you are using HTTP 1.1, then yes, the connection is kept alive by default if you are connecting to an HTTP 1.1 server. The server's Connection response header will indicate whether the server is actually keeping the connection open or not.

Also, as stated by someone else, HTTP 1.1 requests MUST have a Host header or else the request is malformed and can be rejected/ignored by the server. HTTP 1.1 has a notion of virtual hosts running on the same IP, so the Host header tells the server which host the client wants to talk to.

HTTP GET (with Header 'Connection: keep alive') over TCP

1 Answers1