-5

so I built a scraper with PHP - curl. So I can build my API and integrate in my application. The problem is that I am getting blocked by the website in just one page.

What my scraper does:

  1. First it goes to the website that have an iframe ( and this iframe have an iframe inside, which I want to get the src of it).
  2. When it get the first iframe url, it sends a curl request to the iframe src, but the problem is that the site is blocking any request from websites other than their website. So I get blocked. Is there any solution to pretend being the URL of this website.

BTW, this is the URL of the website here. You can inspect the website iframe and see what I mean.

Evara
  • 73
  • 7

1 Answers1

0

sorry buddy! you can’t. If it was possible, I would pretend to be your bank and steal your bank credentials! crazy, right?

you can act like a reverse proxy, for example: I am the user and you are the interface. I come to your website, foo.com and search for google.com. you can send a curl to google.com, get the response from google.com and show it to me. I see foo.com address in my browser but actually I’m browsing google.com. and google.com thinks that your server(foo.com) is using it’s site. not me. and if you get a lot of users, google.com will probably block your server as an attacker server.

read about CORS and Reverse Proxy for more information.

MRNafisiA
  • 155
  • 7