1

I made a website where people can post links for other websites and then the backend generates a preview of the link (by using curl and parsing out the open graph tags available on most websites / by picking the first image, html title etc). Now, fine after some tweaking but sometimes I get some kind of rate limit.

Here is one example of a link I want to parse: https://www.facebook.com/HBR/posts/10157131816732787

I can parse it 4 ou 5 times and get a title, image etc but then if I repeat it I get sent to the login page of facebook. How can I avoid this?

I tried to parse the link at https://developers.facebook.com/tools/debug/sharing however it says "Facebook URLs cannot be crawled". So my question is: how am I even supposed to parse those links if they don't even allow it on their debugger?

Is there any kind of API that allows me to get this information without user login? I don't want to parse entire facebook pages, profiles etc, just get a preview for a link that my users might post on the website.

TCB13
  • 3,067
  • 2
  • 39
  • 68

2 Answers2

2

You MUST use the Facebook Graph API if you want to get data of Facebook Pages (or anything else on Facebook), scraping is not allowed.

In order to get data of Pages you do not own, you need to apply for Page Public Content Access: https://developers.facebook.com/docs/apps/review/feature/#reference-PAGES_ACCESS

An App Access Token (without Login) is sufficient in that case.

API Reference for Pages: https://developers.facebook.com/docs/graph-api/reference/page/

andyrandy
  • 72,880
  • 8
  • 113
  • 130
-3

I dont think show.You can crawl post on public group using python selenium and beautiful soup

Tú Trần
  • 9
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Apr 15 '22 at 08:50