0

My VPS is running on Centos 7 with nginx and MySQL 8.0 I have created 2 cron jobs to backup my files and database everyday at a specific time. everything works fine but I'm trying to send those backup files to my backup host. I created a bash file to automatically send files over backup host, but the problem is, it sends ALL the Tar files to the remote destination, and what I want, is to send only the newly created file to be uploaded each time and do the same routine for the last 7 files (weekly cycle). I ran in to problems and I was told it's better to make the cron job do all the work. Is it possible to tell the cron command, when done creating Tar file (for both sql and files), send them to backup host, and only keep 7 backups by replacing the old one? here is my bash script:

#!/bin/bash
ftp=****
username=****
passwd=****
remote=/DESTINATION on Backup Host/
folder=$1
cd /local folder to pick files/$folder
pwd
ftp -n -i $ftp <<EOF
    user $username $passwd
    mkdir $remote/$folder
    cd $remote/$folder
    mput *
    close
EOF

and here is my cron job:

00 01 * * * mysqldump -u "location_to_username_and_password" "db name" | gzip -c > /location/db_name.`date +\%a\-\%Y\.\%m\.\%d`.sql.gz
00 03 * * * tar -cvpzf /location/db_name.`date +\%a\-\%Y\.\%m\.\%d`.tar.gz -C /route_to_folder/ folder_name
samoolix
  • 31
  • 3
  • 1
    I strongly suggest you consider using something other then ftp for file transfer. Perhaps rsync. Though you might also want to consider a different backup tool like borgbackup. – Zoredache Apr 21 '21 at 07:59
  • My experience with FTP is negative. Consider a 1-second overhead for each and every command or file sent or received. That is `mput *` with 3600 tiny files will take 1 hour. Presumably, the tar turns that into a single file? That would probably be tolerable. – Rick James Apr 21 '21 at 16:08
  • Thank you for your help guys. But don't I need to have ssh access or access to port 22 on the backup host in order to use Rsync? – samoolix Apr 22 '21 at 07:40
  • I don't know what your backup host offers, but FTP is the worst possible choice. Pick any other protocol. The good one for Linux would be NFS, but you could also use CIFS. – Michael Hampton Jun 07 '21 at 20:28

1 Answers1

0

You can use a versioned archiver to store all dumps, without the need to rotate backups, via for example zpaqfranz https://github.com/fcorbelli/zpaqfranz or the original zpaq.

Supposing to mysqldump -uroot -ppippo mydb >/tmp/databasedump.sql just a

zpaqfranz a /whereismybackup.zpaq /tmp/databasedump.sql

Then you can upload the backup with rsync, to a remote server, with something like (of course you need the key of the user for ssh upload)

rsync -I --append --omit-dir-times --no-owner --no-perms --partial --progress -e "ssh -p 22 -i /thekeyoftheuser" -rlt --delete "/whereismybackup.zpaq" "userofthekey@server-ip:/home/userofthekey/thebackup"

And yes, you need ssh (port 22 or whatever) and rsync on the remote server. In this case, using zpaq archive, you do not need to send everytime the entire file, but only the appended bytes (much, much faster)

Franco Corbelli
  • 81
  • 3