3

Recently I have started to use Scrapy on a regular basis to analyze sites which demand the latest browser (user agent) for their content to show up. Now, this may seem like an old time problem, yet up-to-date the issue is quite open. Why?

There is no simple API or Package to generate/download the latest version user agents (in any OS/platform).

A number of packages try to resolve this:

  1. shadow-useragent - but it relies on voluntary server which is for some reason inactive as of now.
  2. latest-user-agents - but it also uses a hosted json file + it also lists old user agents, and it doesn't have any documentation.

Lastly there is this web: www.whatismybrowser.com which is very helpful but can't be easily automated...

Any clue how to resove that?

rubmz
  • 1,947
  • 5
  • 27
  • 49

1 Answers1

6

An old answer, but I was actually also looking for this feature.

Your 2nd option latest-user-agents works as it gets the JSON file from a daily updated source from the same author.

The reason the README.md contains old user-agents is that the repository is not updated, so this is just an example. I tried it today, and it has all the latest user-agents what I could see.

Matteus Magnusson
  • 334
  • 4
  • 8