RSync scripts

[dreamline]

Hi All,
I just finished my RSync backup script and incase somebody might want to know how to go about that, here's my backup script and exclusion list.

First the backup script:


#!/bin/sh
#
# (c) 2010 dreamline
#
# Crontab:
# 30 11,22 * * * <path to script>/rsync_backup.sh > /home/<username>/backup_log.txt
#
#
# Some RSync parameters explained:
# a = archive
# b = make backups
# E = preserve executability
# h = output numbers in a human-readable format
# v = verbose
# z = compress
# l = keep symlink
# L = copy symlink reference
# r = recurse into dirs
# m = remove empty dirs
# n = dry-run
# p = keep permissions
# t = keep modification times
# g = preserve groups
# o = preserve owner
# u = skip files that are newer on the receiver

SRC="/"
DST="/home/<username>"

/usr/bin/rsync -lavzrmpu --force --delete --delete-before --delete-excluded --exclude-from <path to script>/rsync_backuplist.txt "$SRC" "$DST"
chown -R <user>:<user> $DST # Here I chown everything to the chrooted user
chmod -R 744 $DST # Only give full access to chrooted user, others get read-only


Here's my exclusion/inclusion script:


#####################################################################
#
# (c) 2010 dreamline
#
####################################################################
# Backup complete /etc directory
####################################################################
+ /etc**

####################################################################
# Backup MySQL and Admin domains
####################################################################
+ /home
+ /home/mysql**
+ /home/admin**

####################################################################
# Backup DirectAdmin (do not backup mysql.conf because it contains
your da_admin password, also exclude updates and custombuild)
####################################################################
+ /usr
+ /usr/local
+ /usr/local/directadmin
+ /usr/local/directadmin/conf
- /usr/local/directadmin/conf/mysql*
- /usr/local/directadmin/custombuild*
- /usr/local/directadmin/updates*
+ /usr/local/directadmin/**

####################################################################
# Backup php.ini (PHP)
####################################################################
+ /usr/local/lib
+ /usr/local/lib/php.ini

####################################################################
# Backup config.inc.php (phpMyAdmin)
####################################################################
+ /var
+ /var/www
+ /var/www/html
+ /var/www/html/phpMyAdmin
+ /var/www/html/phpMyAdmin/config.inc.php

####################################################################
# Backup complete /root directory
####################################################################
+ /root**

####################################################################
# Exclude all the rest
####################################################################
- *


Hope this helps someone. :) However if you plan on syncing this backup through rsync with your local PC then I suggest you use a chrooted user which is locked in it's own home directory. That's where I create my rsync backup and from there I retrieve it to my windows box using Rsync too by using a public-private ssh key. :) Still testing the windows part of rsync. :)

[mr.applesauce]

How cute...

[SeLLeRoNe]

thanks a lot for sharing your work.

[SeLLeRoNe]

should i suggest you to include custombuild and template customization?

/usr/local/directadmin/data/template/custom/
and
/usr/local/directadmin/custombuild/custom/

for have all the template customization you made on your server.

Ive a question also, where do you put the exclusion list? I mean, in the first script doesnt call the second one, so, how do you backup all?

Thanks

[dreamline]

Well the list here is an include and exclude script. Look closely and you see + and - at the start of the lines. The - is exclude from the backup and + is include in the backup. :)

So the complete DA directory is backed up except for custombuild, updates and mysql.conf.

At the end you see - * which basically says exclude everything from the backup. So if I didn't have any other directives (before the - *) then nothing would be backed up. :)

** means include/exclude directories, subdirectories and files
* include /exclude file or directory (do not recurse into other dirs when used on a directory)

Oh the second script should be named rsync_backuplist.txt. Which is mentioned in the first script by --exclude-from <path to>/rsync_backuplist.txt . :)

[SeLLeRoNe]

yeah thanks i noticed that time later and i didnt post :)

But, on exlcusion list you exlude custombuild
- /usr/local/directadmin/custombuild*

so, this will exclude subfolder aswell, and, the custom subfolder should be nice to backup i think.

Btw, sorry for those idiot questions, but does script are on the server you wanna backup right?

So my question is, this is a local backup? Cause i dont see any ftp/remote connection for uplaod those file, am i right?

Ive read you use another rsync instance on your pc for "import" the backup, but, why dont you upload directly from the script?

Regards

[dreamline]

Hi SeLLeRoNe,
I excluded custombuild because I only use that to build Apache, MySQL and PHP and a few others, however I don't know what's in the custombuild custom folder though, never looked at it. :) If you want to include the custombuild customfolder only then it would look something like this:
+ /usr/local/directadmin/custombuild
+ /usr/local/directadmin/custombuild/custom**
That way the customfolder and all it's content would be backed up too.

I run the script as a cronjob on my server. It then copies the files to a chrooted user on the same server. At home I run Windows, so I had to compile Rsync with cygwin. Then I looked at cwRSync for some settings and needed dll's. Next I created a dos script which connects to my server with the chrooted user and a public-private ssh key. Then it syncs a location on my harddrive.

Now I hear you ask why do you use RSync locally and not from the server. That question is easy to answer: My Server has a fixed IP while my PC at home may change IPs from time to time which might mess up the syncronization if I have rsync on my server connect to my local pc. On my local PC I run a scheduled task to start the windows rsync script which then synchronizes with my server.

Ps. No problem asking questions.


Oh I noticed an other thing from you post:
/usr/local/directadmin/data/template/custom/
The full directadmin folder is included in the backup, only the following files/directories are excluded:
- /usr/local/directadmin/conf/mysql*
- /usr/local/directadmin/custombuild*
- /usr/local/directadmin/updates*
All the other files/directories in directadmin are included in the backup. :)

:)

[SeLLeRoNe]

Thanks again for your explanation.

I would suggest you to use dyndns.org for dynamic ip at home (many routers now support dyndns)
This would permit you to set a host on your server and connect always to your home cause the router that support dyndns will send the new ip everytime it change ;)

I use to do this way to have always my home ip for connect to it.

BTW. ive already added this line to the exclusion list:
+ /usr/local/directadmin/custombuild/custom**
Does this work or your way should work better?

Regards

[dreamline]

Yes custom** is better. :) My bad. :)
I'll have a look at dyndns. :) Thanks.

[SeLLeRoNe]

Im not expert on rsync so i dont really know hwo to set to send via ftp all the data, so, should you please point me somewhere?

Thanks

[dreamline]

Well, actually I haven't gotten into rsyncs feature to sync remote yet. I'm definately not an expert on rsync yet, but in a couple of days I figured out how to use rsync to meet my needs.

However I might dive into remote backing up since I do have 2 servers. I can always see if I can create a cross backup. :) So a backup from server 1 will be stored on server 2 and vice versa. But at the moment I haven't tried that yet. :(

[zEitEr]

Hello,

Rsync is neither using, nor supporting SSL. Am I right? If so, how do you protect your data (of course if you do any protection) while transferring in Internet?

Do you use any VPN or secured tunnels? I've found this article (http://www.bytemark.co.uk/support/technical_documents/backuprsyncssl) and want to get your opinion.

Regards,
Alex.

[mr.applesauce]

rsync can send over ssh which is secured.

[zEitEr]

@mr.applesauce,

thanks for your answer, I'll check it out.