First thanks to Uli for the first step towards some idea’s and Armijn giving good guidelines how to use some variables together with commands.
cause I am a newbie on bin/bash scripting.. but with lots of testing tonight I was able to backup several ‘clients’ content towards buckets and objects on Amazon S3. Probably this is a first good step towards the usage of a simple script for the backup of your server’s content to Amazon S3
Needed: Python, S3cmd, data and the correct settings
note: this script is not finished yet as I am manually executing it and the removal of data from Amazon S3 is not well tested (it works, but I need to check it during the time the stuff is running.
#!/bin/sh
###################################
### ###
### BACKUP FOR
You can take a look on article about backing up an Plesk server on Amazon S3: http://sandbox.frontline.ro/2008/10/31/plesk-backup-to-amazon-s3-testing-phase
As today, I think JungleDisk is the most reliable solution for mounting an S3 bucket.
I’ll update my blog with some info about my implementation with JungleDisk.
@sorin,
the idea’s is nice, but as I have multiple ‘clients’ and domains I want them to have them as clients in my buckets. I succeeded now with my scripts and crontab settings ;-)
Its a shame that there isn’t yet an S3 option within the backup manager on Plesk.
I currently use s3cmd and pleskbackup to achieve a backup per domain
Each domains backup is in a single tar
No scheduled backups are required in plesk control panel
To restore: an ftp user is set up on localhost with a home directory of /home/backups – the backups then show up in the ftp repository on the control panel.
All backups have the same date stamp – set when the cronjob starts – the %H%M format could be removed…
Only backs up domains with physical hosting (ie domain has a directory in /var/www/vhosts )
I currently have this line in crontab… on ubuntu hardy 8.04, Plesk 9.3 and s3cmd 0.9.9.91
/bin/ls -ls /var/www/vhosts | /usr/bin/awk -v mydate=$(date +%Y%m%d%H%M) ‘/\w+\.\w+/ {system (“/opt/psa/bin/pleskbackup domains-name ” $10 ” –skip-logs –output /home/backups/backup_” $10 “_” mydate “.tar”)}’; /usr/bin/s3cmd –delete-removed sync /home/backups/ s3://mybucketname; /usr/bin/s3cmd du | /usr/bin/awk ‘NR==1 {print $1/(1024*1024*1024)” Gb used, ” 5- $1/(1024*1024*1024) ” Gb remaining”}’
to do…
1. Use a default S3 configuration unless a specfic one exists for a domain (or client) – allowing domains to back up to specific S3 accounts – making the
best use of the free tier
2. Stop using s3cmd sync – reducing disk space required on server. Delete all but the latest backup from the server.
3. Change backup_ prefix to daily_ weekly_ or monthly_ dependant on whether day matches last day of month or a sunday or not.
prefix=”daily_”; [ `date +%w` -eq ‘0’ ] && prefix=”weekly_”; [ `date -d tomorrow +%d` -eq ’01’ ] && prefix=”monthly_”;echo $prefix
# I’m sure there’s a better way to do this – but this works…
4. set up deletion of backups – so that I have a backup for the last n days, n weeks & n months – i.e I’d like to keep the last 5 days backups and keep the backups from Sunday for the last 5 weeks and keep the backups from the last of each month for the last 5 months… resulting in a rolling set of 15 backups for each domain. Maybe have n variable dependant on (backup quota for s3 account)/((average size of daily backups for the s3 account)*3)
5. check integrity of each backup file after upload to S3
6. report an exception if size of backup is more than 5% different to previous backup.
7. report an exception if S3 account’s quota is exceeded.
8. and all the other things that I wish were already in Plesk
thanks for this info Jason!
indeed shame that it is not available from plesk
note: my script needs modification when used under 10, cause 10 has different setup between users/resellers/clients