posts tagged as rsync

ssh and rsync

Monday, 29 March 2010 7:42 am by noel
posted in tech | tags: , , ,

i use rsync to backup files within the local network and also through the internet. rsync, by default, does not encrypt the data it transmits so to backup files via the internet one has to encrypt the data or the port that the data travels in using another software. for this purpose i use ssh (secure shell) to create an encrypted “tunnel” between the transmitting and receiving computers.

the conventions i’ll be using: remote means the server where i will be copying files from and local means the server where i will be copying files to. local is also where the rsync backup script is located and initiated.

my assumptions are that both remote and local servers are running linux and both have rsync and openssh installed.

when initiating a backup run from the local server, the remote server would normally ask for a password. this is obviously not good especially when i need to schedule unattended backup runs at odd hours of the day (or night). so for the script not to ask a password i need to generate a public/private pair of keys on the local server to be used with ssh.

to generate a public/private key pair, log in to the console in the local server:

$ ssh-keygen -t dsa -b 2048 -f /home/localbackupuser/local-rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /home/localbackupuser/local-rsync-key.
Your public key has been saved in /home/localbackupuser/local-rsync-key.pub.
The key fingerprint is:
94:87:e5:c3:d0:06:e4:09:3a:76:a2:d2:d7:9b:2e:cc localbackupuser@local

so now we have keys to use to authenticate between the local and remote servers. we now have to copy the contents of the local-rsync-key.pub into the authorized_keys file of the remote server (/home/remoteuser/.ssh/authorized_keys)

i would normally mount the remote server’s drive using sshfs (secure shell filesystem) and edit the authorized_keys file as if it was in my workstation. another way to do this is via a remote console.

for added security, you can limit the computer(s) connecting to the remote server by specifying the ip address of the local server along with the contents of the public key generated above (details here). this would be very useful if the ip address of the local server doesn’t change. unfortunately mine does.

next step is to test the backup script on the local server via ssh. if the backup script starts syncing with the remote server then all that is left to do is add and entry in crontab to automatically start the backup at the time you specify.

links:
using rsync and ssh
sshfs

  • Share/Bookmark
-->

no comments

scripting rsync

Tuesday, 9 February 2010 6:45 am by noel
posted in tech | tags: , ,

here’s the script i use on secure backup runs using rsync—with a little modification to the names and ip addresses for security reasons—but everything else is essentially the same. i have to run this through cron because the server is headless—no monitor. i just access it via the ssh console or browser from another computer.

#!/bin/sh

DATETMP=`date +%Y.%m.%d`
RSYNC=/usr/bin/rsync
SSH=/usr/bin/ssh
# the private ssh key of the local computer
KEY=/rsyncuser/.ssh/id_local
RHOST=remote.ip.addr.ess
RPATH=/remotedata/
LPATH=/localdata/
LOGFILE=/rsyncuser/rlog.$DATETMP.log
# contains the file extensions of files to be excluded from the backup
EXCLUDES=/rsyncuser/localexcludes
OPTS="--exclude-from=$EXCLUDES"

# check if rsync is already running
RUN=`ps x | grep rsync | grep -v grep | wc -l`
if [ "$RUN" -gt 0 ]; then
echo rsync already running
exit 1
fi

$RSYNC -avz -e "$SSH -i $KEY" $OPTS $RHOST:$RPATH $LPATH >> $LOGFILE

i leave an entry in crontab to run the script once each day. sometimes a backup run goes longer than 24 hours so i needed to check if rsync is already running in the server. if the script doesn’t check, it will run another instance of the script and would slow down the server or, worse, brings it down completely.

i have to encrypt all data that gets transferred between the two computers via ssh just in case a naughty third party is “listening in”. i use 2048-bit encryption. processing the data—encrypt at remote; decrypt at local—is a little slower but i am more confident that the data will be safe from eavesdroppers.

i use the exclude-from option to exclude files that shouldn’t be backed up—music and movies—or else the backup will take too long especially on just a dsl line.

if you notice anything wrong with the script, please leave a comment.

note: this is repost from my old blog.

  • Share/Bookmark
-->

1 comment

tiny changes in jaunty

Monday, 27 April 2009 5:14 pm by noel
posted in tech | tags: , , , , , , ,

i noticed a couple of things that changed when i ugraded to ubuntu 9.04

i’m just wondering how it performs with older computers (pentium 4, 256 meg ram). if you know, please leave a note.

  • Share/Bookmark
-->

no comments