0

I am trying to web scrape Instagram accounts to find out how many posts they have (visible on all public and private profiles). I am writing a script to do so.

I tried using the requests library to obtain the HTML code, but when running this below:

import requests

url = 'https://www.instagram.com/instagram/'
r = requests.get(url)
print(r)
print(r.text)

I keep getting the 429 too many requests error even though I have sent less than 10 total (I am manually running the script). I also tried following https://stackoverflow.com/questions/22786068/how-to-avoid-http-error-429-too-many-requests-python#:~:text=Writing%20this%20piece,your%20bot%200.1'%7D) and using requests.get(link, headers = {'User-agent': 'your bot 0.1'}) but that also does not work.

hashfdfasf
  • 31
  • 3
  • 1
    They don't like to be scraped, although you can get around it - for examples by [using selenium](https://pastebin.com/VLZ2vPYK) or [requesting via ScrapingAnt](https://pastebin.com/5ibz2F6p) [ see [sample output](https://i.stack.imgur.com/bLgvJ.png) ]...but there are several python modules for scraping Instagram like [instagramy](https://www.geeksforgeeks.org/scrape-instagram-using-instagramy-in-python/) / [instascrape](https://pypi.org/project/insta-scrape/) / [snscrape](https://github.com/JustAnotherArchivist/snscrape) / etc - any of those should be faster – Driftr95 Jan 09 '23 at 04:32

0 Answers0