Google App Engine - jsoup.get() urlfetch timeout exception

Question

I am trying to run a simple get request using jsoup in my google app engine project. The code is

Document doc = Jsoup.connect("http://www.indianrail.gov.in/")
                      .userAgent("Mozilla")
                      .timeout(60000)
                      .get();

However, when I deploy my app to app engine and run it, it returns an exception as shown below.

com.smartapps.saseatavailability.RailwayResource readPnr: 
Timeout while fetching URL: http://www.indianrail.gov.in/
java.net.SocketTimeoutException: Timeout while fetching URL: http://www.indianrail.gov.in/
    at com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:178)
    at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:43)
    at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:543)
    at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:422)
    at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getResponseCode(URLFetchServiceStreamHandler.java:275)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:443)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:424)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:178)
    at org.jsoup.helper.HttpConnection.get(HttpConnection.java:167)
    at com.smartapps.saseatavailability.RailwayResource.readPnr(RailwayResource.java:229)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_$1.run(Method_.java:171)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_.privilegedInvoke(Method_.java:167)
    at com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_.invoke_(Method_.java:125)
    at com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_.invoke(Method_.java:44)
    at org.apache.wink.server.internal.handlers.InvokeMethodHandler.handleRequest(InvokeMethodHandler.java:63)
    at org.apache.wink.server.handlers.AbstractHandler.handleRequest(AbstractHandler.java:33)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.CreateInvocationParametersHandler.handleRequest(CreateInvocationParametersHandler.java:54)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.FindResourceMethodHandler.handleSubResourceMethod(FindResourceMethodHandler.java:183)
    at org.apache.wink.server.internal.handlers.FindResourceMethodHandler.handleRequest(FindResourceMethodHandler.java:110)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.FindRootResourceHandler.handleRequest(FindRootResourceHandler.java:95)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.HeadMethodHandler.handleRequest(HeadMethodHandler.java:53)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.OptionsMethodHandler.handleRequest(OptionsMethodHandler.java:46)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.handlers.SearchResultHandler.handleRequest(SearchResultHandler.java:33)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.log.ResourceInvocation.handleRequest(ResourceInvocation.java:92)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.internal.log.Requests.handleRequest(Requests.java:76)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26)
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22)
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:63)
    at org.apache.wink.server.handlers.AbstractHandlersChain.run(AbstractHandlersChain.java:48)
    at org.apache.wink.server.internal.RequestProcessor.handleRequestWithoutFaultBarrier(RequestProcessor.java:207)
    at org.apache.wink.server.internal.RequestProcessor.handleRequest(RequestProcessor.java:154)
    at org.apache.wink.server.internal.servlet.RestServlet.service(RestServlet.java:119)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:37)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:260)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:78)

Interestingly, this code works for all other websites I have tried "http://www.facebook.com", "http://www.yahoo.com", etc. Also, the code works fine for "http://www.indianrail.gov.in/" when I deploy it locally and test on my localhost.

Why would this be happening? Is URLFetch intentionally blocking a few websites?

score 0 · Answer 1 · answered Nov 12 '15 at 23:21

0

Timeout exceptions are thrown when connecting to the website takes longer than the specified timeout value. It may be the case that "http://www.indianrail.gov.in/" takes longer than other sites like Facebook to load. Please try increasing the timeout value to something like 60*1000.

answered Nov 12 '15 at 23:21

Joel Min

3,387
3
19
38

The timeout is already set as 60,000. I have also tried increasing it to 300,000 but doesn't help and I receive the exception before the timeout expires. – tbag Nov 12 '15 at 23:26
Oops sorry didn't see it there. Hopefully [this link](http://stackoverflow.com/questions/30630560/google-app-engine-connection-request-timeout-error) and [this link](http://stackoverflow.com/questions/7261567/the-sockettimeoutexception-is-frustrating-on-google-app-enginejava-workaround) would be of help to you. Seems like a IP blocked issue. – Joel Min Nov 12 '15 at 23:33
Don't think its that either because the apache library can access the website from google app engine. – tbag Nov 12 '15 at 23:42

Google App Engine - jsoup.get() urlfetch timeout exception

1 Answers1