0

Let's say that I need to retrieve the HTML of a website making a Httpget in my app. But, if the site has a mobile version of the content, I want this version and not the usual/desktop version (for instance, if you access http://techcrunch.com/ in your mobile you will get a diffent html that you will get if you access this website on your desktop)

I have made the following code:

HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
request.setHeader("User-Agent", System.getProperty("http.agent"));
HttpResponse response = client.execute(request);

BufferedReader br = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
    sb.append(line);
}
String html = sb.toString();

but after that, the html string contains the "desktop" version of the website and not the HTML for the mobile version. My intuition says that I only need to configure the "User-Agent" for the httpget, but it seems that I need to configre anything else. Any help?

Renato Lochetti
  • 4,558
  • 3
  • 32
  • 49

2 Answers2

2

I have solved the problem.

My mistake is in the line

request.setHeader("User-Agent", System.getProperty("http.agent"));

To get the correct user agent I need to create a Webview and do the following:

request.setHeader("User-Agent", webview.getSettings().getUserAgentString());

This way the HTTPGet will retrieve the mobile version of the website content.

Renato Lochetti
  • 4,558
  • 3
  • 32
  • 49
1

I think your on the right track since most sites look at the user agent to determine if the requested page should be the full or mobile version. I think your best bet would be to set the User Agent string to a hard coded value, perhaps just to test at first ? Maby something like:

Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30

John
  • 3,512
  • 2
  • 36
  • 53
  • Thanks for the tip @John but, when calling `System.getProperty("http.agent")` in my mobile it returns `Dalvik/1.6.0 (Linux; U; Android 4.1.2; GT-I8190 Build/JZO54K)`. It should work for this user agent, right? I'll try your hardcoded user agent. – Renato Lochetti Apr 04 '13 at 18:51
  • I figured it might be something like that, probably not the issue then I think usually they just look for 'Android' in the string with regex. Perhaps techcrunch does something special. Other ways they use to detect mobile devices is to look at the supported mime types or the device width. – John Apr 04 '13 at 18:54
  • The user-agent that you have wrote worked. Now, I need to discover the proper way to get the device user-agent and put it in the httpget (If you think that the wabview does it properly, there must be a way to do that) – Renato Lochetti Apr 04 '13 at 19:00