How do I fetch a webpage with node-fetch?

Question

The following code:

import fetch from 'node-fetch'
;(async function fn() {
  const response = await fetch('https://learnitmyway.com/learning-material-software-development/')
  const data = await response.json()
  console.log({ data })
})()

results in: UnhandledPromiseRejectionWarning: SyntaxError: Unexpected token < in JSON at position 0

That is why we had documentation to review. https://developer.mozilla.org/en-US/docs/Web/API/Response/text or https://fetch.spec.whatwg.org/#ref-for-dom-body-text%E2%91%A0 — epascarello, Oct 06 '21 at 15:16
From my point of view it can't be expected that every developer knows where to find this in documentation, especially not in a spec. — David, Oct 06 '21 at 15:19
You learn.... that is how it works. 22 years ago when I started, I had to look at books and pray I found it. It is in the specification. That is the second link! — epascarello, Oct 06 '21 at 15:20
I agree that you can learn it and I also believe that Stack Overflow is here to speed up the process. From my point of view it's not fair to assume that all developers know how to understand documentation or a spec — David, Oct 06 '21 at 15:24
I think it's fair to assume that people who have this problem in the future may not have understood the documentation. — David, Oct 06 '21 at 15:29
SO should generally be used as a last resort after you've exhausted all the other options. Like @epascarello I started around 20 years ago and we didn't have the wealth of documentation that we have now. If you don't understand your code, and haven't studied what's wrong with it, or looked at some documentation, "speeding up the process" does nothing to increase your knowledge of the language. All we can do is point you to the documentation that you didn't read. — Andy, Oct 06 '21 at 15:38
I think it's fairly common for people to get stuck and use Stack Overflow without finding their answer in the documentation, which is why I thought the question was worth posting. — David, Oct 06 '21 at 15:40
So the question remains why you think that you'd be getting JSON as a response. — Andy, Oct 06 '21 at 15:42
Let's not go off topic. From my point of view, it doesn't matter how I came to the mistake. I think this answer sums up my thoughts quite well https://meta.stackoverflow.com/a/299411/5955911 — David, Oct 06 '21 at 15:48

score -2 · Answer 1 · answered Oct 06 '21 at 15:12

Replace json with text as follows:

import fetch from 'node-fetch'
;(async function fn() {
  const response = await fetch('https://learnitmyway.com/learning-material-software-development/')
  const data = await response.text()
  console.log({ data })
})()

How do I fetch a webpage with node-fetch?

1 Answers1