1

A series of sixty HTTP POST's to a PHP file located on my Godaddy server repeatedly cause a SocketException (when using Java JRE6/JRE7), NoHttpResponseException (in Android) or occasionally a SocketTimeoutException (both Java & Android) on the 61st try, and the server becomes unresponsive for approximately 60 seconds at which point I can sneak through a few more requests before the same exception occurs.

java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source) at sun.net.www.http.HttpClient.parseHTTP(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at Tester.postRequest(Tester.java:47) at Tester.main(Tester.java:20)

After spending a week trying to isolate the cause from my Android project, I eventually simplified it into the self-contained standalone Java testcase shown below. In this case, the loop executes correctly 60 times (I parse the response and see the PHP file saw the POST), then fails on the 61st try. It's very repeatable, although I can adjust the number downwards by adding code into the PHP file (such as logging to a file, etc.). Note, that doing HTTP GET doesn't cause an issue, only POST.

import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class Tester
{
    private static final String SERVER_PHP_URI="http://allucanapp.com/phptest.php";
    private static final String CHARSET="UTF-8";

    public static void main(String[] args)
    {
        try
        {
            for (int i=0; i < 200; i++)
                postRequest(i);
        } catch (IOException e) { e.printStackTrace(); }
    }

    private static void postRequest(int count) throws IOException
    {
        String query="command="+count;

        HttpURLConnection conn=(HttpURLConnection) (new URL(SERVER_PHP_URI).openConnection());

        try
        {
            conn.setReadTimeout(10000);
            conn.setConnectTimeout(15000);
            conn.setDoInput(true);
            conn.setDoOutput(true);
            conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
            conn.setFixedLengthStreamingMode(query.length());

            BufferedOutputStream output = null;
            try
            {
                output = new BufferedOutputStream(conn.getOutputStream());
                output.write(query.getBytes(CHARSET));
                output.flush();
            } finally { if (output != null) try { output.close(); } catch (IOException e) { e.printStackTrace(); } }

            InputStream response = conn.getInputStream();

            if (conn.getResponseCode() != HttpURLConnection.HTTP_OK)
                System.out.println("Error "+conn.getResponseCode()+": "+conn.getResponseMessage());
            else
            {
                BufferedReader reader = null;
                try
                {
                    reader = new BufferedReader(new InputStreamReader(response,CHARSET));
                    for (String line; (line = reader.readLine()) != null;)
                        System.out.println(count+":"+line+"\n");
                }
                finally { if (reader != null) try { reader.close(); } catch (IOException e) { e.printStackTrace(); } }
                response.close();
            }
        }
        finally { conn.disconnect(); }
    }
}

And for reference, here's the PHP file. The server supports PHP version 5.3.

<?php
echo "hi there " . $_POST['command'];
?>

I've spent several days looking at this, found numerous other people posting similar types of problems dating back years, generally going unresolved. I've tried the following, all exhibiting the same failures.

  1. I tried both Apache HttpClient (using DefaultHttpClient) and HttpURLConnection.
  2. I paced out the requests, about a second apiece with only minimal improvement (eg. it failed on the 65th iteration instead of 61st)
  3. Upon an error, if I wait slightly more than 60 seconds (a coincidence ?), I can get a few more POST's through before another error.
  4. I played with all the parameters for HttpUrlConnection.
  5. I played with all the various HTTP header settings.
  6. I commented out the code pertaining to the BufferedOutputStream above, ie. I did an empty POST.
  7. I modified the PHP file to log incoming requests. It saw the 60 successful ones, but never received the 61st request (the failing one).
  8. Saving the best for last. I ran this code on both my desktop (wired internet) and laptop (wifi connected to the same router).
    • a) If I run on one, it fails after 60 iterations, then if I run on the other machine, it immediately fails.
    • b) If I run on both concurrently, both run ~30 iterations, when one fails, the other immediately fails.
    • c) If I switch my laptop to run on 3G so it doesn't share the router, while (a) still occurs, (b) does not, they are treated independently by the server.

This problem does not manifest when using my local (10.0.2.2) Zend server. If I didn't know better, I would think the Godaddy server is throttling, maybe some anti-denial-of-service tactic. I'll contact Godaddy, but I couldn't find any php.ini settings that controlled this sort of behavior except for limiting concurrent requests. What troubles me is that this issue has been lurking in my codebase for 2.5 months and only manifested because I was doing performance testing.

To fend off comments about why I should want to do repeated POST's, etc., keep in mind the above is a test case, the original application is using POST's to upload image files onto the server, and while I could merge these to reduce the number of requests, that's just a kludge and creates new issues (limitations on file upload size, etc.)

SuperDave
  • 41
  • 1
  • 3

2 Answers2

1

As it turns out, my testcase was being flagged as a DDoS event by their servers which would temporarily blacklist my client's IP address. Their claim was that 6 attempts within a minute was deemed a threat. I guess I was "lucky" I could do 60 requests in a minute. Their solution ? Upgrade to their virtual private server costing about 6 times as much per month where they didn't think this limitation existed as I'd be responsible for securing the server. Not exactly the most helpful response. I guess their security solution is catered to people clicking on web forms, not to Android apps using PHP to interface to a MySQL database. More annoying was the lack of documentation given that I lost a week debugging the above issue.

Anyways, instead, I'm going to attempt the kludgey route of grouping together all my HTTP POST requests into a very small number, for example, one massively long multi-part request for uploading a hundred images. Then figure out how to deal with tracking intermediate upload/download progress, avoiding buffer overflows, inherent PHP limits, etc., and hope I don't hit some other Godaddy server limitation (such as PHP script timeouts !).

SuperDave
  • 41
  • 1
  • 3
0

It sounds to me also as if they are throttling you requests after a certain amount of requests/time, this may be as you mentioned a case of them preventing potential DDoS attacks against their servers, or possibly that they have not allocated a very high client thread count for serving requests.

If there is a low number of threads available to deal with your requests and they are being constantly connected to through your efforts then they will backlog and eventually time-out causing your lack of response.

Brilliant post with lots of detail, nice to see what you have tried, I would suggest contacting GoDaddy as I'm pretty sure its a limitation on their part.

Let me know how it pans out, good luck

Ben Maxfield
  • 635
  • 7
  • 21