0

I'm trying to write up a bash program that will display the full contents of this website (https://www.trivianerd.com/random-trivia-generator) and redirect it into a text file. I expect the content of the text file would look the same as if we were to open the Developer Tools in Inspect Element mode on the website above, in a browser.

Here is my code so far:

#!/bin/bash

# URL to fetch data from
url="https://www.trivianerd.com/random-trivia-generator"

output_file="trivia3.txt"
curl "$url" > "$output_file"

echo "Data fetched and saved to $output_file"

However, I'm not able to obtain the entire html file as displayed when I open the Developer Tools in Inspect Element mode. In particular, I couldn't locate any mentions of the question and the choices for the multiple choice question are within the html code that I retrieved through my function, but I was able to locate them when I opened the Developer Tools in Inspect Element mode.

Any suggestions on what I should do?

Deezel
  • 111
  • 6
  • 4
    Is there a chance, that the questions are generated/fetched by JavaScript? If so, there's your problem: Bash doesn't execute JavaScript. Chrome (or any other browser) does. – Refugnic Eternium Aug 14 '23 at 11:28
  • _"However, I'm not able to obtain the entire html file as displayed when I open the Developer Tools in Inspect Element mode"_ No, because this does not represent the actual HTML downloaded and is a view of the page after any scripts have been run. If you View Page Source though it's likely to be the same as you've managed to download. – phuzi Aug 14 '23 at 12:02

0 Answers0