How to get all posts by topics from Reddit by search using Web Scraping?

Question

I tried to use Selenium and Beautiful Soup to scrape all posts mentioning certain topics. Although I scrolled down to the bottom of the search page, I only got a portion of the results because only about 250 posts were returned. Does anyone know what the issue is?

Here is my code below:

keyword = 'spx'
base_url = "https://www.reddit.com/search/?q="
driver.get(f"{base_url}{keyword}")

time.sleep(3)

previous_height = driver.execute_script('return document.body.scrollHeight')

while True:
    driver.execute_script('window.scrollTo(0, document.body.scrollHeight);')
    time.sleep(3)

    new_height = driver.execute_script('return document.body.scrollHeight')

    if new_height == previous_height:
        break
    previous_height = new_height

source = driver.page_source

soup = BeautifulSoup(source, 'html.parser')

posts = soup.find_all("faceplate-tracker", {"source": "search",
                                              "action": "view",
                                              "noun": "post",
                                              "data-testid": "search-post"
                                              })

for post in posts:
    subredditList.append(post.find("a", 
                          {"class": "flex items-center text-neutral-content-weak font-semibold"}).text.strip().split('/')[1])
XXXXXXXXXXX