NPM Request not get html from thai url. (Web Scraping)

Question

This code will not show html in console it show error

var request = require('request');
var thai_url = "http://pantip.com/tag/Honda_(มอเตอร์ไซค์)";

request(thai_url, function (error, response, html) {
        if (!error && response.statusCode === 200) {
            console.log(html);
        } else {
            console.log("Error");
        }
});

But I change thai_url from มอเตอร์ไซค์ to %E0%B8%A1%E0%B8%AD%E0%B9%80%E0%B8%95%E0%B8%AD%E0%B8%A3%E0%B9%8C%E0%B9%84%E0%B8%8B%E0%B8%84%E0%B9%8C

Like This

var request = require('request');

var thai_url = "http://pantip.com/tag/Honda_(%E0%B8%A1%E0%B8%AD%E0%B9%80%E0%B8%95%E0%B8%AD%E0%B8%A3%E0%B9%8C%E0%B9%84%E0%B8%8B%E0%B8%84%E0%B9%8C)";

request(thai_url, function (error, response, html) {
        if (!error && response.statusCode === 200) {
            console.log(html);
        } else {
            console.log("Error");
        }
});

The second code working right. Can I change มอเตอไซค์ to %E0%B8%A1%E0%B8...(more)? Or If it's possible, How am I doing for http://pantip.com/tag/Honda_(มอเตอร์ไซค์) working with npm request ? Thank you for any help.

score 0 · Answer 1 · answered Aug 20 '15 at 09:21

0

First one is working fine. Just log the error instead of "error" string.

Make sure your IDE supports utf character. otherwise on saving it might be changing it. That why you might get error.

answered Aug 20 '15 at 09:21

Hiren S.

2,793
15
25

I use Netbean. I change console.log("Error"); to console.log(error); . It show null. I open this file with notepad++ & notepad. it use utf-8 encode for this file. – Tor Thailand Developer Aug 20 '15 at 09:27
Actually i tried with Webstrom and it prints HTML. So no problem in the code. – Hiren S. Aug 20 '15 at 09:29

score 0 · Accepted Answer · answered Aug 20 '15 at 10:11

0

Update. Now I can solve this problem by this

var thai_url = encodeURI("http://pantip.com/tag/Honda_(มอเตอร์ไซค์)");

Use encodeURI. http://www.javascripter.net/faq/escape.htm

answered Aug 20 '15 at 10:11

Tor Thailand Developer

1
3

NPM Request not get html from thai url. (Web Scraping)

2 Answers2