Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [wget]

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

GNU Wget (or just Wget, formerly Geturl) is a computer program that retrieves content from web servers, and is part of the GNU Project. Its name is derived from World Wide Web and get. It supports downloading via HTTP, HTTPS, and FTP protocols.

Its features include recursive download, conversion of links for offline viewing of local HTML, support for proxies, and much more. It appeared in 1996, coinciding with the boom of popularity of the Web, causing its wide use among Unix users and distribution with most major GNU/Linux based distributions. Written in portable C, Wget can be easily installed on any Unix-like system.

Source: wikipedia

Man page

290 questions
2
votes
1 answer

How would I write a terminal command to download a folder with wget from a Media Temple (gs) server?

I'm trying to download a folder using wget on the Terminal (I'm usin a Mac if that matters) because my ftp client sucks and keeps timing out. It doesn't stay connected for long. So I was wondering if I could use wget to connect via ftp protocol to…
user41157
  • 189
  • 2
  • 9
2
votes
2 answers

mirror a site with wget and download static media

I'd like to mirror a site with wget and convert all the links to the local copies I've downloaded. So far that's easy all I have to do is wget -mk http://site.com However all of the static media is located in a different domain, if I follow all of…
mountainswhim
  • 121
  • 1
  • 2
2
votes
0 answers

Is there a tool I can use to test out multiple vpn tunnels in one machine?

I use wget -q -O https://checkmyip.com/csv to determine if I am connected to a VPN but I want to verify multiple VPN tunnels on the same machine. Is there a tool that can allow me to send a command that will allow me to load a csv/json file from…
LeanMan
  • 181
  • 4
2
votes
2 answers

Can't serve static files with ampersand in name

I have a website mirrored with wget and I'd like to serve it via Apache. The problem is that I can't access files with special chars. For example, I have files with names like this: $ ls -N components/com_flexicontent/librairies/phpthumb/ | head…
Grzegorz Adam Kowalski
  • 121
  • 3
1
vote
4 answers

download files from text file that contain files list under linux

i have a text file that contains a list of urls for files of my word for example : http://domain.com/file1.zip http://domain.com/file2.zip http://domain.com/file3.zip http://domain.com/file4.zip ...etc how can i batch download all files to a folder…
EzzDev
  • 113
  • 4
1
vote
1 answer

PASV FTP fails for wget/ftp but works for curl/lftp

I have a server running Centos 7 and vsftpd-3.0.3 (installed from source to get the latest version, though everything I describe was also applicable to the vsftpd-3.0.2 version that was previously installed). vsftpd is configured to use passive…
dan
  • 323
  • 1
  • 5
  • 16
1
vote
1 answer

Upgrading wget on Ubuntu 16.04 server

I'm trying to upgrade my copy of wget on the server. Currently its running: 1.17.1-1ubuntu1.4 The problem is that I need 1.19 so I can get a new feature: http://tomszilagyi.github.io/2017/02/Wget-retry-on-http-error My patch has been part of the…
Andrew Newby
  • 1,102
  • 2
  • 25
  • 58
1
vote
0 answers

How to use wget with redirect and long URL?

I am trying to download the latest release of a web application from GitHub using wget on the shell. As far as I remember, I never had any trouble with wget. I issue the following command: wget…
Matthias Ronge
  • 467
  • 1
  • 7
  • 18
1
vote
0 answers

How to set http_proxy env variable and have wget using it transparently before login (Debian)

By setting environment variable http_proxy="http://proxy-server" I'm able to call programs like wget, curl, etc, using the set http proxy transparently. To persist this to the whole system I've added http_proxy to the /etc/environment. I'm using…
filipe
  • 111
  • 4
1
vote
0 answers

Start script are not cleared table of processes

I have a proxy server, all ok. But, i have some problems & questions. First: when i make "/etc/init.d/nginx stop" or "/etc/init.d/nginx restart", of my config, it not reload and all nginx process are not killed. My script: case "$1" in start) echo…
Piduna Valeriu
  • 37
  • 8
1
vote
1 answer

Changing all HTML absolute links to relative

I have a website that has a bunch of absolute addresses, and I need to move it (the whole contents of that website) up a level, so all of the absolute links need to be converted to relative. I know about wget with --convert-links, but it doesn't…
BubbleFish
  • 13
  • 1
  • 3
1
vote
3 answers

Running wget as cron job

I want to download backup archives from one server to another. The server has a script that delivers the latest backup archive, the file has specific name for each date. I want wget to preserve that name. When I run: wget -q --content-disposition…
johny332
  • 33
  • 1
  • 1
  • 3
1
vote
2 answers

vshpere wget: bad address 'releases.ubuntu.com'

on a raw vsphere esxi 6 without any vm installed i have to setup vm's and ... remotely so i decided to open ssh to esxi and download an Ubuntu iso using wget this way: wget http://releases.ubuntu.com/16.04/ubuntu-16.04-server-amd64.iso but when i…
Leo
  • 31
  • 1
  • 4
1
vote
0 answers

Can wget detect remote duplicate files (with different names) and download only one?

I'm retrieving a dynamic website contents recursively. Unfortunately, files are available from different dynamic URLs. For instance, maybe file http://foo.bar/bla.php?q=xyz and http://foo.bar/bla.php?q=abc are exactly the same (they have the same…
Silas
  • 121
  • 3
1
vote
1 answer

How do I ensure that a CoreOS cloud-config service is able to download files?

I am defining a one-shot service in my CoreOS cloud-config, but it is failing due to not being able to download files from Google Cloud Storage (via wget): Apr 13 11:09:56 staging-node-ys9y.c.experimentalberlin.internal sh[1132]: Connecting to…
aknuds1
  • 2,095
  • 3
  • 17
  • 23
1 2 3
…
19 20