0

I'm trying to hit an URL and upon hitting that I will hit consecutive URLs.

Both in Postman and Ruby (I'm using NET HTTP), I'm getting an error saying, "enable Javascript for website to function properly".

But in browser, everything works fine. In browser, if I turn off javascript, I see the same error.

Question: Is it possible to simulate same browser experience in Postman or Ruby NET:HTTP ?

sofs1
  • 3,834
  • 11
  • 51
  • 89

1 Answers1

0

No, it's not possible. Not with NET::HTTP. You actually need to run the browser. But you can run it in a headless mode!

Fortunately, in Ruby there are some great gems that allow you to achieve that. Myself, I most often use Ferrum, which runs Chromium in the background and allows you to interface with the website using an extended Nokogiri API. Ferrum uses the CDP protocol to interface with Chromium and possibly Firefox.

In the past I have used Watir which basically wraps Selenium WebDriver with a nicer API and works with all the browsers. Your mileage may vary with this, I had a lot easier time with Ferrum and Ferrum provides a Nokogiri-compatible API, while Watir doesn't. On the other hand, Watir is a lot more mature.

I have tried puppeteer-ruby, which is mostly a port of the JavaScript puppeteer library. Like Ferrum, it also uses the CDP interface, but it may provide a better support than Ferrum for various features, because as a port of puppeteer, it contains a lot of knowledge that was put into puppeteer which is widely used. It's quite new, but very interesting.

hmdne
  • 434
  • 2
  • 5