0

I am working on a research project and I need to be able to scrape the hashtags from a large amount of Tiktoks. Tiktok has a lot of measures in place to counter the normal methods of web scraping, but I have found that I can download the HAR file from chrome and extract the hashtags from there.

Every HAR to JSON python or java module I have been able to find either does not work, or does not have the hashtag information in the output JSON file. Most online HAR viewers only display the headers and params while I need the raw JSON file to be able to access the hashtags. I am able to access this from this chrome extension but with the large amount of data I need to view it crashes.

I need to get the raw JSON for multiple web pages out of a single HAR file.

Any help is appreciated!

ThisPotatoGuy
  • 1

1 Answers1

0

*** Disclaimer: Answer by HttpWatch Developer ***

Our HttpWatch software (Windows only) that works with Chrome and Edge can help you do this. It can handle multiple pages and if there's not enough information in the HAR format you could use the HttpWatch log file (HWL). The HWL files contain every detail of each HTTPS transaction.

Also, HttpWatch has a programmatic interface (see https://apihelp.httpwatch.com) that could be used to search for and extract hashtags.

HttpWatchSupport
  • 2,804
  • 1
  • 17
  • 16