HttpClient 3.1 setting connection timeout slows down the response

Question

In my swing application, I am using HttpClient 3.1 to perform Rest call. Most of the POST methods calls are taking 4-5 seconds to respond. I am using MultiThreadedHttpConnectionManager. I increased the default max connection but still no use. When I commented out the following connection timeout code, the response is coming in within 1 seconds

httpClient.getParams().setParameter("http.connection.timeout", 5 * 1000);

Not sure why the response time varies when I comment out the above line. Is this a right way? Correct me if I am wrong

FYI

HttpConnectionManager connMgr = new MultiThreadedHttpConnectionManager();
HttpConnectionManagerParams connMgrParams = new HttpConnectionManagerParams();
connMgrParams.setSoTimeout(30 * 1000);
connMgr.setParams(connMgrParams);
if (proxyPort.equalsIgnoreCase("8888")){
    httpClient.getHostConfiguration().setProxy(DEFAULT_HOST, 8888);
}
httpClient = new HttpClient(connMgr);
httpClient.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler(0, false));
httpClient.getParams().setParameter("http.connection.timeout", 5 * 1000);