1

I am having some issues with the npm request package and getting it to call any URL properly unless the URL is to a server that is off. Example, I can't call amazon.com but I can call dev.personalDomain.com which points to an EC2 instance that is off entirely.

I tried to strip out the application logic to make it so that just the error is produced.

For this example assume monitor.body.url = 'amazon.com' is a string

My code

request({url: `http://${monitor.body.url}`, timeout: 20}, function (error, response, body) {
    console.log(`Response: ${response}`);
    console.log(`Body: ${body}`);
    console.log(`Error: ${error}`);
    if(error == 'ENOTFOUND' || error == 'ETIMEDOUT') {
      response.statusCode = 500;
  }
  if(error || response.statusCode < 200 && response.statusCode > 299 || !response.statusCode) {
    if(!response || !response.statusCode)  response.statusCode = 500;

    // Application logic

  }
  // If nothing went wrong just console.log -- This is used for debugging not useful in prod
  if(!error || response.statusCode > 299 || response.statusCode < 200)console.log(`Successfully called ${monitor.body.url}`);

});

Error message

Response: undefined
Body: undefined
Error: Error: ESOCKETTIMEDOUT
/home/ec2-user/environment/service/app.js:91
          if(!response || !response.statusCode)  response.statusCode = 500;
                                                                     ^

TypeError: Cannot set property 'statusCode' of undefined

I went looking on google/SO and came across that it might be DNS causing the issue and to test/resolve that I use:

const dns = require("dns");
dns.setServers(['8.8.8.8', '1.1.1.1']);

This made no change on the result of the request from above.

Does anyone have a suggestion or idea on what this might be? I am having no luck finding an example that is close and produces the same error message.

EDIT**

My DNS is perfectly fine on my dev machine so zero issues there. Also I didn't have this issue when I had just request({'http://${monitor.body.url}', function (error, response, body) { so seems like the error with how I am using request

joshk132
  • 1,011
  • 12
  • 37
  • I'm not sure `setServers` impacts anything except calls using the DNS library directly. Most Node code will use the default resolver regardless of settings there. Does the equivalent fetch with another tool like command-like `curl` have any problems? – tadman Mar 10 '19 at 02:27
  • @tadman Care to explain what you mean? How would I go about setting the DNS servers for `request` then? I found little mention of the DNS settings inside the npm page for request. – joshk132 Mar 10 '19 at 02:28
  • I'm not sure if the `request` library uses the Node.js DNS facility or it just leans on the default (e.g. [`gethostbyname`](http://man7.org/linux/man-pages/man3/gethostbyname.3.html) or equivalent in LibUV), so that's why I'm suggesting you should verify that you can resolve that hostname in something other than Node first. – tadman Mar 10 '19 at 02:29
  • > I'm suggesting you should verify that you can resolve that hostname in something other than Node first I have no problem going to `amazon.com` if that is what you are suggesting. As mentioned in the OP assume that the url I am trying to reach is `amazon.com` so based off that and that I can go to amazon.com and can also use curl to get to it I am going to say I am safe and can resolve the DNS outside node. – joshk132 Mar 10 '19 at 02:32
  • Based on some testing it looks like `request` really does not care what your `setServers` are set to. You'll need to get your machine's DNS operational or do the resolving and make a request to a fixed IP address instead with no DNS involved. – tadman Mar 10 '19 at 02:32
  • Try `example.com` or `stackoverflow.com` to see if those load at all or not. Narrow the problem down from "maybe it's Amazon, or maybe it's my local application firewall". – tadman Mar 10 '19 at 02:33
  • @tadman It does not matter what the URL is, it will not work at all except in how I layed out in the OP. If I set it to `dev.personalDomain.com` which is pointed to a EC2 instance that is off/stopped then it works but for all other domains it does not work. – joshk132 Mar 10 '19 at 02:36
  • @tadman Also ready my OP edit – joshk132 Mar 10 '19 at 02:37
  • Are you getting errors back from `request`? If so, what are they? Note that using the promise-driven approach either through `then` or `await` forces you to pay attention to errors. – tadman Mar 10 '19 at 02:38
  • @tadman Check my OP put it is there from the initial post. Check the part that says about the error message. – joshk132 Mar 10 '19 at 02:39
  • I put 3 console.logs in my code to show the results. In my error section I include the output of those console.logs – joshk132 Mar 10 '19 at 02:39
  • Ah, I see that now. What does it do with `example.com`? – tadman Mar 10 '19 at 02:42
  • @tadman Exact same as it does with `amazon.com`. I tested again after you original ask and exact same result I got earlier when I tested it. – joshk132 Mar 10 '19 at 02:43
  • @tadman I am 99% sure it is not DNS in any which way unless it is DNS inside Node.js/ my application. – joshk132 Mar 10 '19 at 02:43
  • @tadman Thanks for your help, and check my answer for the actual issue. – joshk132 Mar 10 '19 at 02:54

1 Answers1

2

Figured out the issue. My edit gives and an idea of what it might be... So the timeout I am using with Request is actually in milliseconds not seconds like I thought it was. So it was timing out after 20 milliseconds which was resulting in an error where not even the DNS was finishing properly. I changed it to actually be 20 seconds and the issue seems resolved.

joshk132
  • 1,011
  • 12
  • 37