Apache Http Client - Stop Removing Double Slashes from URL

Question

I am using Apache HTTP Components Client 4.5.7 to request a URL that containing a double slash. When I look at the wire log, I see that the double slash was "fixed" to be only one slash. Unfortunately that is no desireable behaviour for me, as it will cause the request to fail.

Background: I am requesting resized images from Thumbor (an image resize server). A Thumbor URL will basically look like this:

this url will cause thumbor to download http://host.com/image.jpg and resize it to fit into 200x200 pixels.

The code looks like this:

HttpGet httpUriRequest = new HttpGet("http://thumbors-server/usafe/200x200/http://host.com/image.jpg");
CLIENT.execute(httpUriRequest, responseHandler); 

What httpclient sends to the server. however is this:

DEBUG o.a.h.headers     http-outgoing-1 >> GET /unsafe/300x300/http:/host.com/image1.jpg HTTP/1.1 
DEBUG o.a.h.headers     http-outgoing-1 >> Host: localhost:4002 
DEBUG o.a.h.headers     http-outgoing-1 >> Connection: Keep-Alive 
DEBUG o.a.h.headers     http-outgoing-1 >> User-Agent: Apache-HttpClient/4.5.7 (Java/11.0.1) 
DEBUG o.a.h.headers     http-outgoing-1 >> Accept-Encoding: gzip,deflate 

please note that http://host.com was replaced by http:/host.com (notice the missing second /). This will cause the request to fail.

How can I stop http client from "fixing" the url that I pass to it?

Answer 1

For a similar case I encountered, the best solution was to use URLEncoder.encode to url-encode the embedded URL.

In your example,

new HttpGet("http://thumbors-server/usafe/200x200/" + URLEncoder.encode("http://host.com/image.jpg", "UTF-8"))

Answer 2

I encountered this today in the http-client v4.5.13. The server I'm hitting/testing does not handle URI's properly so URLEncoding didn't work. I needed a workaround (short of using raw sockets) in my code.

After some debugging found that you can disable URI normalization using the RequestConfig.custom().setNormlizeUri() method when setting up the client configuration.

Javadoc link to the RequestConfig builder

Example:

requestConfig = RequestConfig.custom()
                    .setConnectTimeout(connectionTimeoutMS)
                    .setConnectionRequestTimeout(connectionTimeoutMS)
                    .setSocketTimeout(connectionTimeoutMS)
                    .setNormalizeUri(normalizeUri)
                    .build();

Answer 3

Problem is in URIUtils.rewriteURI() where is this code:

final StringBuilder buf = new StringBuilder(path.length());
boolean foundSlash = false;
for (int i = 0; i < path.length(); i++) {
    final char ch = path.charAt(i);
    if (ch != '/' || !foundSlash) {
        buf.append(ch);
    }
    foundSlash = ch == '/';
}
uribuilder.setPath(buf.toString());

So double slashes in uri path are always replaced with one slash. You can use another http client like OkHttp which doesn't make this normalization.