10

I'm having trouble using wget for my Debian 7.0 VPS server hosted by OVH.

I'm trying to download a ZIP file from MediaFire, and when I connected via SSH I typed,

wget http://download1472.mediafire.com/5ndlsskkyfmg/dgx7zbbdbxawbwd/Vhalar-GGJ16.zip

Then, this is my output,

--2016-03-07 20:17:52--  http://download1472.mediafire.com/5ndlsskkyfmg/dgx7zbbd                                                                         bxawbwd/Vhalar-GGJ16.zip
Resolving download1472.mediafire.com (download1472.mediafire.com)... 205.196.123                                                                         .160
Connecting to download1472.mediafire.com (download1472.mediafire.com)|205.196.12                                                                         3.160|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://www.mediafire.com/?dgx7zbbdbxawbwd [following]
--2016-03-07 20:17:52--  http://www.mediafire.com/?dgx7zbbdbxawbwd
Resolving www.mediafire.com (www.mediafire.com)... 205.196.120.6, 205.196.120.8
Connecting to www.mediafire.com (www.mediafire.com)|205.196.120.6|:80... connect                                                                         ed.
HTTP request sent, awaiting response... 301
Location: /download/dgx7zbbdbxawbwd/Vhalar-GGJ16.zip [following]
--2016-03-07 20:17:52--  http://www.mediafire.com/download/dgx7zbbdbxawbwd/Vhala                                                                         r-GGJ16.zip
Connecting to www.mediafire.com (www.mediafire.com)|205.196.120.6|:80... connect                                                                         ed.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `Vhalar-GGJ16.zip'

    [  <=>                                  ] 94,265       440K/s   in 0.2s

2016-03-07 20:17:52 (440 KB/s) - `Vhalar-GGJ16.zip' saved [94265]

The download took less than 1 second, and it's a 280MB zip file. Also, it seems to say "440 KB/s", and that math just doesn't add up.

I'm confused as to why I can't download this zip file to my server via SSH, instead of downloading it to my computer, then re-uploading it to the server.

Does anyone see a flaw I'm making in my command?

Logan Butler
  • 159
  • 1
  • 2
  • 9

4 Answers4

9

What you're doing when you're using wget to download that zip file is just downloading the html page that the zip file sits on. You can see this because if you redo the command to output to an html file like such:

wget http://download1472.mediafire.com/5ndlsskkyfmg/dgx7zbbdbxawbwd/Vhalar-GGJ16.html

and open it in the web browser of your choice, you'll get the fancy html page of that link with the mediafire download button on it.

This is entirely because mediafire wants you to verify that you're human with a captcha before you can download it. Try doing the captcha and then issuing the command:

wget http://download1472.mediafire.com/gxnd316uacsg/dgx7zbbdbxawbwd/Vhalar-GGJ16.zip

It will work.

If you have not completed the captcha on whatever computer you're trying to download it from, you need to. This is what the captcha originally looks like. Once you finish it and click "Authorize Download" you'll have free reign to wget the file from the server.

Mediafire Captcha

If all else fails, download it originally on your computer and use the scp command to transfer it over.

Chirality
  • 745
  • 8
  • 22
  • Okay, I reloaded the page and did the capacha, and it still did the same thing. (downloaded the small, html/zip file) – Logan Butler Mar 08 '16 at 01:54
  • When you open `http://download1472.mediafire.com/gxnd316uacsg/dgx7zbbdbxawbwd/Vhalar-GGJ16.zip` or click on it on the server, do you get any error message or does the file transfer start? It's weird you cannot write to your server. – YouHaveaBigEgo Mar 08 '16 at 01:54
  • Specifically for mediafire downloads, you'll need an api key seen here. https://www.mediafire.com/developers/getting_started/getting_started/ – Chirality Mar 08 '16 at 01:54
  • There is no issue with writing files. Did no one bother to check what captchas are on the site? The captcha is preventing a direct download. Either try to set up the development version of mediafire or use a different file sharing service. – Chirality Mar 08 '16 at 01:56
  • When I go to the page on my server (www.example.com/zip.zip) it just downloads the zip, the small HTML filled one. – Logan Butler Mar 08 '16 at 01:57
  • Much better and more knowledgeable answer than my own. :) – Dewi Morgan Mar 08 '16 at 01:57
  • @LoganButler are you able to open up a GUI on your server or is it command line only? If you can open up a browser and download it on your server or even just long enough to pass the captcha, you'll be fine. If you keep trying to issue the command without completing it, nothing will work and you'll continue on downloading the html page. – Chirality Mar 08 '16 at 02:00
3

Look at the contents of the 94kb file that you downloaded in something like vi. Odds are it's not a zip file, but a html file, telling you what went wrong, and what you need to do to download the file.

A browser would have known this (the mime type would tell it that it is being served HTML, and it would display it to you rather than download it).

It is likely that this is a measure by Mediafire to prevent automated downloads of their files. It's possible that spoofing the user-agent header might help, but unlikely.

Dewi Morgan
  • 1,143
  • 20
  • 31
  • Alright, I opened it up using `nano`, what should I look for inside this file specifically? – Logan Butler Mar 08 '16 at 01:32
  • So what did you SEE? What does it look like? Is it HTML? (first characters are "" or " – Dewi Morgan Mar 08 '16 at 01:37
  • It's about 30 lines of HTML. – Logan Butler Mar 08 '16 at 01:38
  • So when you read the HTML, what error message is it displaying? [edit whooo, pastebin!] – Dewi Morgan Mar 08 '16 at 01:46
  • Looks like it's a webpage asking you for a captcha, along with (amongst other things) the text "Why we use Captchas: We use Captchas to prevent bots from downloading excessively, which steals bandwidth from our users. Captcha enables us to preserve the value of service that is important to our users." So, mediafire doesn't WANT you to download stuff through wget, and the amount of hoopjumping you'd need to do in order to download your file through wget is probably greater than simply getting it to your server some other way. – Dewi Morgan Mar 08 '16 at 01:49
  • I tried downloading the URL you linked through wget using `--user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0"` to see if a different user agent would help - but got the exact same result. So simple user agent spoofing won't work. You'd need (most likely) to either manually type in the captcha message, or register an account, and log in using wget (which can be a giant pain in the butt on some sites, even if it works). – Dewi Morgan Mar 08 '16 at 01:53
  • https://superuser.com/questions/312305/downloading-a-rar-file-from-mediafire-using-wget <- likely related. Particularly the last. Answer is old, though, so may not work any more. – Dewi Morgan Mar 08 '16 at 01:55
  • One answer is simply saying it's against the ToS. The other says one bullet about wget not being able to be beaten by captchas which is @LoganButler's issue – Chirality Mar 08 '16 at 01:58
  • 1
    The last answer on that page worked. It's so weird that just taking the ".zip" part away let it download. – Logan Butler Mar 08 '16 at 01:59
  • 1
    Thanks for the help Dewi. – Logan Butler Mar 08 '16 at 01:59
0

Just one tip for anyone working with wget to download a file, but the URL has a proxy string at the end after the file name (eg.?This_is_a_query_string_sample_&_123545_& ) i.e. the URL is of the form:

http://download1472.mediafire.com/5ndlsskkyfmg/dgx7zbbdbxawbwd/Vhalar-GGJ16.html?This_is_a_query_string_sample_&_123545_&

In this case always use double quotes while using wget (since & has a special meaning in shell environments)

wget "http://download1472.mediafire.com/5ndlsskkyfmg/dgx7zbbdbxawbwd/Vhalar-GGJ16.html?This_is_a_query_string_sample_&_123545_&"
Harman Singh
  • 1
0

You can also download the zip file specifying a new name to that file with the wget option -O.

wget -O new_name_for_the_file.zip <url-address.zip>
claudius
  • 747
  • 1
  • 10
  • 24