Getting Checking your browser before accessing template and 503 status code with https.request(options,callback) node.js

Question

I want to get the html of this page for parsing(click the link to understand what content i want to get).

Here's my code to request this page content

    var https = require("https");

    var fs = require("fs");

var options = {
    hostname: "www.prizebond.net",
    port: 443,
    path: "/dlist.php?num=455",
    method: "GET"

};

var response = "";

var req = https.request(options, function (res) {
    res.setEncoding("UTF-8");
    console.log(res.statusCode);
    res.on("data", function (chunk) {
        response += chunk;
    });

    res.on("end", function () {

        fs.writeFile("750-bond.html", response, function (err) {

            if (err) {
                console.log(err.message);
            }
            console.log("File downloaded");

        });
        console.log("end");

    });
});

req.end();

Now the problem is that in my 750-bont.html file, I am getting the weird the result of "Checking your browser before accessing the prizebond.net" not the original content. Here's the screenshot what I got when I open the 750- bond.html file in browser.

What I am doing wrong? And how can I get the original content of this webpage?

What are you doing wrong? You're trying to crawl a website that don't want to be crawled — Alon Eitan, Apr 29 '17 at 07:47
I think I should wait for 5 seconds and then proceed the request? — Aamir, Apr 29 '17 at 07:55

score 0 · Answer 1 · answered Oct 04 '18 at 13:59

You can't, unless you write something more sophisticated, but you probably shouldn't.

The purpose of Cloudflare-protection is to prevent what you are trying to realize unfortunately.

You could look into a possibility to access whatever you want to access by a public API or something that prizebond.net provides for example.

Getting Checking your browser before accessing template and 503 status code with https.request(options,callback) node.js

1 Answers1