1

What I want to accomplish is to know what the exact rules on Youtube (Youtube-sutdio) are for webscraping en using bots to automate data gathering for some channels (around 20). Youtube-studio in particular. In there terms and conditions I read the following:

"Access the Service using automated means (such as robots, botnets, or scrapers), except (a) when using a public search engine in accordance with the YouTube robots.txt file, (b) with YouTube's prior written consent obtained or (c) as permitted by applicable law."

So I went to: https://studio.youtube.com/robots.txt

Robots.txt from youtube-studio

When I read the file it seems to me that Youtube studio allows crawlers with a crawl delay of 0.5 seconds. So that means I can write a crawler that gathers information from the Youtube studio right? Since there are no url's disallowed. Maybe its an obvious question since I gave the answer myself, but I just wanted to check what you guys opinions were. Because I don't have a lot of experience with crawlers and it is very important that Youtube doesn't ban my IP or worse. Can you guys take a look and tell me if you think?

Groupa
  • 53
  • 6
  • 1
    You are right, you are allowed to crawl this website. But make sure you stick to the delay given! ^^ [This might help](https://stackoverflow.com/questions/36405998/is-this-robots-txt-syntax-with-an-empty-disallow-correct) – DevConcepts Oct 02 '21 at 13:43

0 Answers0