Downloading Text Files Using Powershell

Question

I'm trying to download and save the text file http://www.gutenberg.org/cache/epub/164/pg164.txt using Powershell. I tried using the code:

$curl http://www.gutenberg.org/cache/epub/164/pg164.txt -OutFile verne.txt

But instead of saving the text file it saved the text file for the page source of http://www.gutenberg.org/ebooks/164?msg=welcome_stranger. I'm wondering if there's something wrong with my code or if I need to use another code.

Answer 1

It's a redirect. If you put the url in the browser you will get the same welcome stranger page. My guess is that they don't want you to access this content in this way. They may require log in, or at the very least, a valid session cookie.

Answer 2

Your link is a redirection, try this :

$uri = 'www.gutenberg.org/ebooks/164.txt.utf-8'
$request = Invoke-WebRequest -Uri $uri -MaximumRedirection 0 -ErrorAction Ignore
 
if($request.StatusDescription -eq 'found')
{
   #redownload the new url (redirection)
   $request=Invoke-WebRequest -Uri $request.Headers.Location
   $request.ParsedHtml.body.outerText
}