-1

It's been 3 days now, 3 days looking and trying different alternatives of request-promise (deprecated), So I really hope someone would be able to help me here.

I'm making my first scraping server app, So I was using request-promise bust since it got deprecated I'm looking for something else, But I need it to support unGziping, As you may know request-promise have this feature witch comes in handy when scraping a lot of data.

I liked axios & node-fetch but when I use 'Accept-Encoding: gzip, deflate, br', All I get is some coded data that does even get decoded on some websites like : http://www.txtwizard.net/compression

So My Question Is : Is There A Way To Use Gzip with Axios or Node-Fetch ???

smart design
  • 71
  • 6

1 Answers1

0

You can use got with decompress: true.

const { body } = await got('http://www.example.com', { decompress: true });

decompress defaults to true so you don't need to pass it.

Although axios/fetch should decompress by default, so maybe there's something else going on with your request.

You can always decompress it yourself using zlib built-in module

Marcos Casagrande
  • 37,983
  • 8
  • 84
  • 98