Posts Tagged ‘backup’
Today I tried Backup software of CloudBerry backup. What is so special about this backup tool?
For months I am using CloudBerry Explorer for S3 (I have to shame myself not register the software yet) to see what has been stored in my buckets at Amazon’s S3 service. Weekly a backup of my webserver is made and with some scripts I made the stuff is transferred to the Amazon S3 Cloud storage service. but I have not a good backup for my Windows 7 machine yet. Data is stored to my NAS devices but not yet in the cloud. And when it is in the cloud I can access it whereever I am. Pricing is fair where I have unlimited storage. Around 10 to 20GB a month seems to cost me around 5 to 8$ I believe.
So Cloudberry made CloudBerry Backup software able to run under Windows to tranfer local data to Amazon. Personally I do not think many people have or use Amazon S3 other than that it is being used in one or another kind of service like ‘dropbox’. But I am techie enough, pay by CreditCard to Amazon and a happy S3 user. So Cloudberry backup is very very suitable for me, and probably for you.
Note: users of Dunkel or Walrus can use it as well (next to Amazon S3)
starting the software and you can enter configure. Very convenient is the option to import your Amazon S3 settings from other Cloudberry software.
After Importing the data you can select backup profiles which are already defined or you select what kind of data you want. Excluding certain file types is possible. Storing data with some compression or with an encrypted password is possible as well and not to forget: get a notice when the backup fails or is finished.
Of course it is possible to remove files from the backup if the original file has been deleted as well. So purging has also a function with the CloudBerry Backup Software
And of course you can see and follow the whole process. The backup will take some time depending on how much MB’s or GB’s needs to be transferred. The local upload bandwidth is limiting the speed. This is unfortunately not very fast for many.
and last but not least: Cloudberry has now also implemented an only bucket explorer so that you can access your data through a website rather than that this is necessary through an application. Minder nice and less graphical implemented is this also possible at Amazon Right now
First thanks to Uli for the first step towards some idea’s and Armijn giving good guidelines how to use some variables together with commands.
cause I am a newbie on bin/bash scripting.. but with lots of testing tonight I was able to backup several ‘clients’ content towards buckets and objects on Amazon S3. Probably this is a first good step towards the usage of a simple script for the backup of your server’s content to Amazon S3
Needed: Python, S3cmd, data and the correct settings
note: this script is not finished yet as I am manually executing it and the removal of data from Amazon S3 is not well tested (it works, but I need to check it during the time the stuff is running.
#!/bin/sh
###################################
### ###
### BACKUP FOR DOMAIN ###
### ###
###################################
DOMAIN=put-here-a-client-name
BASEDIR=/var/lib/psa/backupscript
cd /$BASEDIR
# $ENVIRONMENTAL VALUE: use basedir for the backups (note my master reseller is called ‘beheer’)
DATADIR=/var/lib/psa/dumps/resellers/beheer/clients
# $ENVIRONMENTAL VALUE: save logfile
LOGS=”/var/lib/psa/backupscript/log`date +_%d%m`.log”
# $ENVIRONMENTAL VALUE: value to xx days, so keep earlier days, can be used daily and modified after testing
REMOVE=$(python s3cmd ls s3://bucketname | grep `date –date=’14 days ago’ +%d%m%Y` | cut -f 4 -d ‘/’)
# first backup the new stuff
python s3cmd put -r –progress -v $DATADIR/$DOMAIN/domains/ s3://bucketname/”$DOMAIN”_`date +%d%m%Y`/ >> $LOGS
echo $REMOVE date will be deleted
# delete the object older than (xx) days defined
python s3cmd del s3://bucketname/$REMOVE/*
rm -rf $DATADIR/$DOMAIN/domains/*
in this script I first define the environmental variables to make sure you only need to enter the correct path and ‘client’ name
than the content on your server is put into the bucked on S3, Then old backup removed from S3 and the actual backup deleted from your server to safe space.
Note: probably I need to use a ‘sync’ option after the put to make sure all data is correctly put onto the server (kind of validation?) if all is uploaded correctly
there is not an if than else statement, the backup content on server is ALWAYS removed when stuff fails ..
for a first script a nice try .. I know the bucketname could also be a ‘variable’ but did not choose for that (yet).
update 7/01/10
added
BASEDIR=/var/lib/psa/backupscript
cd /$BASEDIR
Otherwise s3cmd cannot work ;-)
I am a newbie in bin/bash scripting so help is appreciated
I am using s3cmd with Amazons S3 service
what I want is to copy per day/week/month some backup files. this is working. My files are copied to:
s3://bucketname/[folder], where folder = date (04012010 as example (date +%d%m&Y)
Listing with s3cmd shows indeed the uploaded folders
python s3cmd ls s3://bucketname
DIR s3://bucketname/03012010/
DIR s3://bucketname/04012010/
the script as far is this:
#!/bin/sh
V_DATA=/var/lib/psa/dumps
V_LOGS=”/var/lib/psa/backupscript/log`date +_%d%m`.log”
echo “Starting sync…”
python s3cmd sync –progress -v $V_DATA/ s3://bucketname/`date +%d%m%Y`/ >> $V_LOGS
note: –progress -v is optional to see in shell what is happening
my knowledge is too less to grab the ls content, count the last x days/weeks (and keep the last x days/weeks *as example*) and remove the oldest folder through the simple command:
python s3cmd del s3://lendhosting/[03012010]/*
File s3://bucketname/test deleted
ANY ANY help is really appreciated!
Using a Conceptronic Mediagiant and you want to save your settings before a firmware upgrade? Here is how you can do it
A forced firmware upgrade is deleting all settings, but with a script it is possible to save the information. (and possible it will allow you to do more things).
1. Copy the shooting.sh file to an USB Flash Drive (use link to save the file)
2. Insert USB Flash Drive into the MediaGiant
3. Goto the Copy page and focus on the shooting.sh file
4. Push on your remote the GOTO button and than push the REPEAT button  (you see 2x the invalid icon) the configuration is being backupped.
5. The device will be rebooting, after the reboot than you can do the firmware upgrade.
6. When firmware upgraded is finished, please go into the Copy page and focus on shooting.sh again
7. push on your remote control the GOTO button thanpush REPEAT button the configuration is restored, device will reboot
Please be noticed: the script is creating a ‘VenusSetup.dat’ file on your HDD, you have to REMOVE after your settings are restored.Cause if exist a next time it will use that file for your settings. Even if you have made changes afterwards
rsync -avz -e ssh [username]@www.domain.com:/path/to/get/data/from/ /path/to/mirror-location/
I’m not really happy as it’s still a little unknown to me how to do this unattended (storing authorized keys is a little abacadabra for me) ..
Message to self: to have all data from servage retrieved directly to the new server this following command can be used. You need SSH on your new server.
lftp -u ‘[username],[password]‘ ftp.servage.net -e “mirror /path /destination/path new server”
This command needs to be entered in a ssh session. As we speak this command is already downloading my data for about 12 hours now ..
My backup FTP script is NOT GOOD.. it exhausted my memory ?? on the webserver caushing the server to ‘crash’ / not react for 4 hours, even connecting my monitor to the server: I could type my username, but login: nope ..
So message to self: how the hell can I FTP 40GB of data via a shell script without exhausting my memory .. <sigh> …
rebooting (fingers always crossed) and after 5 minutes the server was up again
#!/bin/bash
cd /home/backup/server
if [ -e data ]
then
cd /home/backup/server/data/
HOST=’ftp-server-address’
USER=’username’
PASSWD=’password’
ncftp -u $USER -p $PASSWD $HOST << EOT
binary
cd /Volume_1
mkdir `date “+%d-%m-%y”`
cd `date “+%d-%m-%y”`
mput *
bye
EOT
fi
cd /home/backup/server
rm -rf data
with some help of Matthijs en of course myself and a little ‘maggi’ I created a bin/bash shell script to backup data (backup of the server) to one of my other FTP server(s)
it checks if the data folder is available and if so than the a ftp session is started, logged on, a folder with the todays date is made and the data is put in that folder. After that the data folder including the content will be removed. Yes I know there is no validation if A = B but I’ve no idea how to accomplish that, but this works in my test environment. This script is saving me about 4 Hours of datatransfer where FTP is showing me a nice 10MB/s where Samba / NFS is showing me a not satisfied performance at all .. (I believe the total amount of data is about 40GB I have to backup everytime … ) ..
hints and tips welcome to have a A=B comparison before removal of the data folder ;-)
General information for users of AroundMyroom, Mixfans.org, Mixfreaks, tapas-recepten, kikkuh.nl. judithsgarden.eu and more.ÂÂ
- BlueQuartz updates. Major update today with 90 updates for BQ.
- Change of POP3 server. BQ upgraded to Dovecot. Had to enable POPS to have it function properly
- Removed dust from Server
- Backup running [takes about 7 hours]
AroundMyRoom updates
- changed permalinks of AroundMyroom old style to new playground style
- changed permanent folder pr0n to erotica
- installed WordPress and changing styles
