I'll be scraping clips from twitch and merging them to create a single video file. I already figured out the scraping of twitch clip links(but i only get 16-20 videos because i need to scroll with selenium but i dont really mind it, if you have a working solution then make an answer about it) and also the simple merging videos.
I'm scraping links with:
#!/usr/bin/python3.9
import bs4
import requests
import time
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
# Initialize driver and run it headless
options = Options()
options.headless = True
driver = webdriver.Firefox(options=options)
def extract_source(url):
agent = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"}
source=requests.get(url, headers=agent).text
return source
def extract_data(source):
soup=bs4.BeautifulSoup(source, 'html.parser')
names=soup.find_all('a', attrs={'data-a-target':'preview-card-image-link'})
return names
driver.get('https://www.twitch.tv/directory/game/League%20of%20Legends/clips?range=24hr')
# I wait 3 seconds for the clips to get pulled in
# I'd like here to scroll down a bit so i can scrape more clips, but even after i tried some solutions my firefox(was debugging in GUI mode, not headless as it is now) wasnt scrolling
time.sleep(3)
extract_links=extract_data(driver.page_source)
for a in extract_links:
print(a.get('href'))
driver.quit()
# I tried scrolling using this but didnt work, not sure why
# this script is supposed to scroll until youre at the end of the page
# SCROLL_PAUSE_TIME = 0.5
# # Get scroll height
# last_height = driver.execute_script("return document.body.scrollHeight")
# for i in range(3):
# # Scroll down to bottom
# driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# # Wait to load page
# time.sleep(SCROLL_PAUSE_TIME)
# # Calculate new scroll height and compare with last scroll height
# new_height = driver.execute_script("return document.body.scrollHeight")
# if new_height == last_height:
# break
# last_height = new_height
I'm joining videos together after downloading(with youtube-dl) with ffmpeg:
ffmpeg -safe 0 -f concat -segment_time_metadata 1 -i videos.txt -vf select=concatdec_select -af aselect=concatdec_select,aresample=async=1 out.mp4
Where videos.txt is as follows:
file 'video_file1.mp4'
file 'video_file2.mp4'
...
I can't really find answers on how to add a watermark(different for each video, although i found this it doesnt explain how to add a unique watermark to individual videos but the same watermark to two videos) without having to render each and every video twice but doing so in one go.
I think I stumbled upon some people who made their videos.txt as follows in purpose of adding extra options to each video:
file 'video_file1.mp4'
option 1(for video_file1.mp4)
option 2(for video_file1.mp4)
file 'video_file2.mp4'
option 1(for video_file2.mp4)
option 2(for video_file2.mp4)
...
Would this work for unique watermarks for each videos(lets suppose watermarks are named video_file1.png, ... meaning the same as the videos, also the watermark is transparent in case that needs more configuration)
