8

I have a 38GB "Moodle" tar file that I need to transfer from the old server (Shared hosting), which we can call "Server A" to the new server (Dedicated virtual), or "Server B". I'm with Mediatemple if that helps.

I've already tarballed the directory and used wget to transfer it to server B via SSH, however - it's a puny 100GB package and I'm at 96% disk capacity - meaning I cannot untar the file on server B! Is there any way I can transfer this huge file from Server A to Server B, preserving permissions and with least possible chance file corruption, which will not make me hit the disk limit?

While I have spent quite a while doing this so far, I would be willing to abandon my original plan if someone can offer a better idea - and I would be very grateful!

alexbass
  • 83
  • 3

2 Answers2

21

Don't tar. Use rsync -av to preserve permissions while transfering the files. Though like tar, this does not preserve selinux context. Not that I would consider that important though.

Dennis Kaarsemaker
  • 19,277
  • 2
  • 44
  • 70
  • 1
    Hi Dennis, thanks for the reply. I don't suppose you could elaborate a bit on how to use the rsync command? Or provide a useful article? Thanks again – alexbass Jul 25 '13 at 20:52
  • 1
    A useful article would be the rsync manpage :) But in a nutshell: if you have ssh access from serverb to servera, run this on serverb: `rsync -av servera:/path/to/files/to/copy/ /path/to/copy/to` – Dennis Kaarsemaker Jul 25 '13 at 20:58
  • 1
    If you're not sure how to use fundamental commands on the OS that you're using, start with the help facility. On Unix machines, that's often something like `man rsync` or `rsync --help` – mfinni Jul 25 '13 at 20:59
  • Super - thanks for that. If I'm transferring a directory however, would I not need to change it to add an `-r` to make it recursive? for instance `rsync -arv ssh root@servera.com /path/to/files/to/copy/ /path/to/copy/to` Many thanks again! – alexbass Jul 25 '13 at 21:08
  • 1
    `-a` implies `-r` (and many more flags people kept forgetting until `-a` was added as "these are the useful flags for making backups") – Dennis Kaarsemaker Jul 25 '13 at 21:12
  • Super - worked like a charm. Thank you very much for your help! – alexbass Jul 25 '13 at 21:25
  • Please edit answer instead of prolongating comments: You may make this more readable ;-) – F. Hauri - Give Up GitHub Jul 27 '13 at 05:38
  • You may have to use `--numeric-ids` option to ensure owner and groups won't change – F. Hauri - Give Up GitHub Jul 28 '13 at 07:46
  • i suggest using rsync over ssh if You have ssh access on both servers. Also You may have sensitive info like db content. `rsync -rav -e ssh user@server:/var/www/prod/ /var/cache/www` – Guntis Aug 02 '13 at 06:35
  • 1
    If security is not a problem (I mean you trust the network between the two machines) nc is also an option. On the server: tar -zc source | nc -l 1313 On the client: nc SERVER_IP 1313 | tar -zx – Tsvetomir Dimitrov Aug 05 '13 at 07:13
12

You probably want to buy more disk space, but assuming you don't, you could...

pipe the tarball around rather than downloading it.

newserver#  ssh olduser@oldserver "cat /path/to/tarball" | tar xf -

or if you don't have SSH access to your old server

newserver# wget -O - http://oldserver/path/to/tarball | tar xf -

or use rsync like Dennis said.

Be creative. There are other solutions I'm not mentioning.

Community
  • 1
voretaq7
  • 79,879
  • 17
  • 130
  • 214
  • You may have to use `--numeric-owner` option before making tarball to ensure owner and groups won't change. `ssh ouser@oserver tar -zcplC /src_path --numeric-owner . | tar -zxpC /dst_path ` – F. Hauri - Give Up GitHub Jul 28 '13 at 07:50
  • @F.Hauri `--numeric-owner` can have (potentially serious) unintended consequences if the UID/GID scheme differs between the two machines. This is usually not an issue in the internal environment, but moving from "shared hosting" to "dedicated virtual" this could cause problems. You should expect to have to fix ownership on the new server if using `tar`, `cpio`, and possibly even `rsync`. – voretaq7 Jul 30 '13 at 16:11
  • Yes, you're right but, not knowing about this kind of option could have (same) border effect too. Best of all: know what you're doing and... rtfm;-) – F. Hauri - Give Up GitHub Jul 30 '13 at 22:50
  • ... some interesting way (filters) to use (or not) to *be creative*: `nc` òr `openssl`, `sed`, `gpg`, `uuencode` and `mail` (with care;) , `enscript` + `lpr` + *[avian carriers] (http://fr.wikipedia.org/wiki/IP_over_Avian_Carriers)* + `scan` + `ocr`... But there are even more... – F. Hauri - Give Up GitHub Jul 30 '13 at 22:57