I am trying to scrape YouTube videos based on a certain (or set of ) keywords. So far I can download only the first page of YouTube videos - roughly 20-30 videos. So far, I have used BeautifulSoup, urllib.request and pytube to do the needful. I am attaching some code here : how would I go about downloading more/all of the videos related to a specific keyword?
# Search through divs
for i in divs:
href= i.find('a', href=True)
urls.append([href.text,"https://www.youtube.com"+href['href']])
video_details = {}
hashtags = []
for link in divs:
for span in link.findAll('span', attrs = {'class':'standalone-collection-badge-renderer-text'}):
for a in link.findAll('a',attrs={'class': 'yt-uix-sessionlink'}):
hashtags.append(a.text.strip())
video_details['HASH_TAGS'] = hashtags
#print(hashtags)
def Downloader(url):
yt = pytube.YouTube(url)
video = yt.streams.first()
video.download('x')
