1

YouTube's "recommended videos" are based on viewing history (i.e. personalization), either from cookies or account history.

When scraping recommended videos using beautifulsoup, is there any personalization?

Note: the beautifulsoup "recommended videos" do not match my own when I manually search, suggesting that my own personalization does not affect the results. But does some other personalization affect the results? 

import urllib.request
from bs4 import BeautifulSoup


goto = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
body = urllib.request.urlopen(goto)
soup = BeautifulSoup(body, from_encoding=body.info().get_param('charset'))
recommended_urls =[]
for link in soup.find_all('a', href=True):
    if "/watch?" in link['href']:
        recommended_urls.append(link['href'])
list(set(recommended_urls))
Martin Gergov
  • 1,556
  • 4
  • 20
  • 29
unicoder
  • 15
  • 9
  • 1
    As you've guessed, there is no personilizationdata since the request doesn't have any cookies. The page should have videos YouTube recommends to unregistered users by default. – t.m.adam Oct 02 '19 at 21:12

0 Answers0