Java communication fails through web proxy using Apache HttpClient

Question

I have a desktop client that sends data to a web server and I can't seem to get through the proxy server.

Update: I am getting a 407 HTTP error when trying to communicate through the proxy.

When downloading information from my web server, everything is fine. Once the user configures the proxy server (using a dialog box I wrote) the download works fine. But uploading data using org.apache.http.client.HttpClient is not working.

I am configuring the proxy server with code like this after gathering the information from a JDialog.

        System.setProperty("http.proxyHost", proxyHost);
        System.setProperty("http.proxyPort", "" + portNumber);

Now, once we do that, simple downloads work fine. For instance, I have code that reads some xml data from my web server (see below). On the customer's network the error in my catch block was displayed before the proxy settings were configured and then everything worked fine once the correct proxy was set.

/**
 * Loads a collection of exams from the web site. The URL is determined by
 * configuration or registration since it is State specific.
 */
public static int importExamsWS(StringBuilder msg) {
    try {
        java.net.URL onlineExams = new URL(examURL);
        //Parse the XML data from InputStream and store it.
        return importExams(onlineExams.openStream(), msg);

    } 
    catch (java.net.UnknownHostException noDNS) {
        showError(noDNS, "Unable to connect to proctinator.com to download the exam file.\n"
                + "There is probably a problem with your Internet proxy settings.");
    }
    catch (MalformedURLException | IOException duh) {
        showFileError(duh);
    }
    return 0;
}

However, when I try to SEND data to the web server it is as if the proxy settings are being ignored and an IOException gets thrown. Namely:

org.apache.http.conn.HttpHostConnectException: Connection to http://proctinator.com:8080 refused

Now I know that port 8080 is not blocked by the customer's web filter because we tested the address in a web browser.

Here is my code for verifying the registration ID entered by the user: Update: Now I am also setting the proxy in this method.

//Registered is just an enum with ACTIVE, INACTIVE, NOTFOUND, ERROR
public static Registered checkRegistration(int id) throws IOException {    
    httpclient = new DefaultHttpClient();
    Config pref = Config.getConfig(); //stores user-entered proxy settings.
    HttpHost proxy = new HttpHost(pref.getProxyServer(), pref.getProxyPort());
    httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
    URIBuilder builder = new URIBuilder();
    String path = "/Proctorest/rsc/register/" + id;
    try {
        builder.setScheme("http").setHost(server).setPort(8080).setPath(path);
        URI uri = builder.build();
        System.out.println("Connecting to " + uri.toString());
        HttpGet httpget = new HttpGet(uri);
        HttpResponse response = httpclient.execute(httpget);
        System.out.println(response.getStatusLine().toString());
        if(response.getStatusLine().getStatusCode()==200) {
            String msg = EntityUtils.toString(response.getEntity());
            GUI.globalLog.log(Level.INFO, "Server response to checkRegistration(" + id + "): " + msg);
            return Registered.stringToRegistered(msg);
        }
        else {
            GUI.globalLog.log(Level.INFO, "Server response status code to checkRegistration: " + 
                    response.getStatusLine().getStatusCode());
            return Registered.ERROR;
        }
    }
    catch(java.net.URISyntaxException bad) {
        System.out.println("URI construction error: " + bad.toString());
        return Registered.ERROR;
    }
}

I'm almost positive that the problem is coming from the proxy server configuration, yet the docs for SystemDefaultHttpClient claim that it takes uses System properties for http.proxyHost and http.proxyPort . I know that the properties have been properly set, but I am still getting this authentication error. Viewing the logs generated from my program showed me this:

checkRegistration INFO: Server response status code to checkRegistration: 407

How do I resolve this authentication error?

How do you send data? a POST? a PUT? a GET with parameters? Sorry, I cannot find out from your question. — Simone Gianni, Jun 28 '12 at 22:49
We are using GET here. Other methods not shown use POST. In the code above notice HttpGet httpget = new HttpGet(uri); — Thorn, Jul 05 '12 at 17:24
Can you set up a proxy in your dev environment, then on your dev machine, use the firewall to block all outgoing http, other than to the proxy? — artbristol, Jul 05 '12 at 18:38
Proxy configuration is outside my skills set. I have Windows Vista and Apache Web server on my dev machine. Is it east to set up a proxy? — Thorn, Jul 05 '12 at 20:14
Squid is easy enough to set up. But you need to set up the proxy on a separate machine, otherwise the firewall config is irrelevant. — artbristol, Jul 06 '12 at 10:07
I am not using JSON. In the example above the registration is just sent as part of the path. Other methods just encode form data - plain text. I also send data by posting xml and csv. — Thorn, Jul 06 '12 at 20:14
Just curious - is the proxy server using basic or NTLM authentication? — ali haider, Jul 08 '12 at 22:56
another question - is there any proxy server information specified in the browser settings? — ali haider, Jul 08 '12 at 22:58
Are you proxying via the server (to another server) or you're using proxy in your application? — tftd, Jul 08 '12 at 23:13
The browser does have proxy configuration and I am using a proxy configuration in my app as shown in the code I posted. — Thorn, Jul 09 '12 at 17:20

ali haider · Accepted Answer · 2012-07-08T23:14:13.913

5

I'll take a shot. If the proxy server is using basic authentication, you could use the following snippet as an example:

DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getCredentialsProvider().setCredentials(
    new AuthScope("PROXY HOST", 8080),
    new UsernamePasswordCredentials("username", "password"));

HttpHost targetHost = new HttpHost("TARGET HOST", 443, "https");
HttpHost proxy = new HttpHost("PROXY HOST", 8080);

httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);

If the proxy server is using NTLM authentication, I do not believe NTLM support is available for all proxy server versions (it will work with some NTLM authentication proxy server versions - check if the proxy is using v1 or v2 of NTLM authentication). For reference & workarounds, you can check the following:

http://hc.apache.org/httpcomponents-client-ga/ntlm.html

http://htmlunit.sourceforge.net/ntlm.html

http://devsac.blogspot.com/2010/10/supoprt-for-ntlmv2-with-apache.html

Depending on your proxy server, you may need to look into NTCredentials instead of UserPasswordCredentials.

I would also suggest that you may wish to look into using wireshark to capture network packets and check the response from the proxy server to be absolutely sure what is causing the problem.

edited Jul 08 '12 at 23:14

answered Jul 08 '12 at 23:08

ali haider

19,175
17
80
149

for confirmation, I would suggest using wireshark to capture network packets. If that takes too long, you can just confirm which proxy server it is (if its a corporate environment, they most likely have the proxy server either in IE settings or point to a script in IE settings or perhaps have a proxy server client running on the desktop to route traffic out). – ali haider Jul 09 '12 at 02:19
Good suggestion, but wireshark is probably over-kill. I confirmed the proxy settings with the IT manager for this network. But he never told me authentication is required. When I tried to go on-line with my laptop I was asked for a username / password for the proxy server. But users logged into the cooperate network are never prompted for this. But it looks like my software will need to prompt for it. – Thorn Jul 09 '12 at 07:56
wondering - is the issue resolved now? sometimes (though mostly not), some corporations can look into providing service IDs or even help bypass proxy servers though I am guessing you would like to solve the problem without it. – ali haider Jul 09 '12 at 23:46
I am hoping to test it this week. It's complicated because I don't have direct access to the network that raised this issue (a public school on Long Island). So I'm waiting for an employee to do the test for me; I already integrated your solution into my code. My inability to test your solution is the only reason I haven't handed you the bounty yet. – Thorn Jul 10 '12 at 05:22
no worries on the bounty - it would be good to keep an eye on the NTLM version they're using, and if so, please share your results as it would be helpful to (i'm sure) many others. – ali haider Jul 10 '12 at 12:53

score 1 · Answer 2 · answered Jul 11 '12 at 17:08

Just for completeness, since this question is found searching for JVM http.proxy properties, if using JVM infrastructure proxy username and passwords can be provided using :

System.setProperty("http.proxyUser",proxyUserName)
System.setProperty("http.proxyPassword",proxyUsePassword).

score 0 · Answer 3 · answered Mar 07 '14 at 15:41

0

For some reason, setting parameters on client didn't work for me.

But worked on http method.

answered Mar 07 '14 at 15:41

Pradeep

97
1
8

Java communication fails through web proxy using Apache HttpClient

3 Answers3

Linked