2

I'm trying to request this website but I keep getting a 400 Bad Request error. This code works for just about any other site I've tried that isn't built with squarespace so I'm guessing that's where the problem is. 

var request = require('request');
var cheerio = require('cheerio');
var url = 'http://www.pond-mag.com/';



request(url, function(error, resp, body){
    if(!error){
        var $ = cheerio.load(body);
        console.log(body);
    }
});
blipblop
  • 347
  • 1
  • 2
  • 14

2 Answers2

5

Figured it out just had to manually set the headers object. Heres the code that fixed it in case anyone else has the problem:

var options = {
url : 'http://www.pond-mag.com/',
  headers: {
   'User-Agent': 'request'
  }
};

Then, just pass the options var to the request instead of the url.

blipblop
  • 347
  • 1
  • 2
  • 14
0

I use Squarespace for a lot of different apps and thought it was worth mentioning that Squarespace has native support for getting the JSON of any Squarespace page. If you append ?format=json to the URL you can pull make the request and get JSON back.

jasonbarone
  • 172
  • 3
  • 17