0

I am new to postgresql. I've connected to my demo database with psycopg2 and inserted data with python faker. I want to take incremental backup with the tool WAL-E and want to see how it actually works. But the few tutorials aren't helping that much as they're not for naive users.

Can someone help explaining the steps simply how to backup data with WAL-E? That would be a great help ! I'm using psql 10.4 in Ubuntu 18.04.

Thanks

Proteeti Prova
  • 1,079
  • 4
  • 25
  • 49

1 Answers1

3

Install Python & Dependencies

apt-get install python-dev python-virtualenv libevent-dev pv lzop daemontools
sudo apt-get install -y python3-pip
sudo apt-get install build-essential libssl-dev libffi-dev
sudo apt-get install -y python3-venv

Create & Activate Virtual Environment

Sudo python3 -m venv /opt/wal-e
source /opt/wal-e/bin/activate

Install Wal-e

sudo /opt/wal-e/bin/pip install wal-e[aws]

Setup Environment Variables

umask u=rwx,g=rx,o=
mkdir -p /etc/wal-e.d/env
sudo -i -u root
echo "Your AWS Secret Key" > /etc/wal-e.d/env/AWS_SECRET_ACCESS_KEY
echo "Your AWS Access Key" > /etc/wal-e.d/env/AWS_ACCESS_KEY_ID
echo 'Your AWS Region > /etc/wal-e.d/env/AWS_REGION
echo 's3://some-bucket/directory/or/whatever' > /etc/wal-e.d/env/WALE_S3_PREFIX
sudo chown -R root:postgres /etc/wal-e.d

Change Postgres Config to Activate Archiving

sudo nano /etc/postgresql/10/main/postgresql.conf
wal_level = replica
archive_mode = on
archive_command = '/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-push %p'
archive_timeout = 60
sudo systemctl restart postgresql

Test Wal-e is Working

sudo -i -u postgres
/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e backup-list

Now Push Your First Base Backup to S3

/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e backup-push /var/lib/postgresql/10/main

Now Set the Cron For Regular Backup

sudo -i -u postgres 

mkdir logs
touch backup.log
touch deletebackup.log

crontab -e

00 00 * * * /usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e backup-push /var/lib/postgresql/10/main > ~/logs/backup.log 2>&1
00 00 * * * /usr/bin/envdir /etc/wal-e.d/env  /opt/wal-e/bin/wal-e delete --confirm retain 1 > ~/logs/deletebackup.log 2>&1
Chandan Singh Gadhwal
  • 187
  • 1
  • 8
  • Hi @Chandan Singh Gadhwal , thanks for posting simple and step by step procedure. However, for step 5, there is no directory named us envdir when I created that manually and try to test wal-e backup. It's throwing below message:/usr/bin/envdir: Is a directory. Could you please help me to troubleshoot this. – Channa Jan 17 '20 at 08:04
  • Hi Shiwangini, Sorry for the delay. You must need to login with postgres user to access /usr/bin/envdir. Also this is not a directory this is executable file created by postgres. First login with this command sudo -i -u postgres as mention above. – Chandan Singh Gadhwal Jan 20 '20 at 07:24