Traven Backups

What things are backed up off site from traven.

Postgres dumps

There is a daily cron job for each site user (eg uk, nyc etc) that does this (in the case of the uk site its called from the dailycron.sh script):

# backup postgres database
/home/uk/bin/dump.sh

That script deletes old backups (to stop the partition filling up), and creates new ones, this is the uk version:

#!/bin/sh
NAME=uk
DATE="`date +"%Y-%m-%d"`"
DIR=/tmp/pg_dump/$NAME

# delete old backups to ensure that the partition
# doesn't fill up
find $DIR -type f -mtime +2 | xargs rm -f

export PGUSER="$NAME"
export PGPASSWORD="`perl -ne 'if (/\s*Database\.Password\s*=\s*(\S+)/) { print "$1\n" }' /mir/uk/setup_uk/etc/config.properties`"

# Portable dump
pg_dump -F p -d "$NAME" | gzip -9 > "$DIR/$NAME-$DATE.sql.gz"

# Quick-to-restore dump
pg_dump -F c -f "$DIR/$NAME-$DATE.pg" "$NAME"

Inc files

Since inc files are not saved to the database these also need backing up, the UK uses has this cronjob to do this and the other site users also have this set up:

# backup the inc files
50 23 * * * /home/uk/bin/inc-backup.sh

Which runs this script:

#!/bin/bash

# this makes backups of the inc directory
# chris 2005-11-18

SITE="uk"
DATE=`date "+%Y-%m-%d"`
BACKUPDIR="/tmp/pg_dump/$SITE"

tar -zcf $BACKUPDIR/$SITE-$DATE.inc.tgz /imc/mir/$SITE/inc/

config.properties

This file is generally not in the CVS since it contains the database connection details. It also contains many site specific customisations so it does need to be backed up. The UK site uses this script to backup this file and it is run in the same cronjob as the inc backup.

#!/bin/bash

# this makes backups of the config.properties file
# since this is not in cvs because it contains the db 
# passwd but it has a lot of essential customisations
# chris 2005-12-23

SITE="uk"
DATE=`date "+%Y-%m-%d"`
BACKUPDIR="/tmp/pg_dump/$SITE"

cp -p /mir/$SITE/mir/etc/config.properties $BACKUPDIR/$SITE-$DATE.config.properties
gzip $BACKUPDIR/$SITE-$DATE.config.properties

rrsync to remote servers

Please only set this up to backup onto encrypted partitions since the database dumps contain sensitive information!

First you need to install the backup machines ssh key on traven (this will have to be a passphrase-less key if it's to be run via cron) and then add this to the start of the key (this means that the key can only be used from one remote ip address and it only has read access to the directory with the backups):

from="backup.server.domain.name",command="/usr/local/bin/rrsync -ro /tmp/pg_dump/" ssh-dss AAA...

Then on the backup machine you run this command via cron:

rsync -e ssh -qaz traven.indymedia.org.uk: /bak/traven.indymedia.org.uk/tmp/pg_dump/

The initial ssh connection will need to be made without the command="" prefix to the ssh key in order that the fingerprint can be accepted / checked. Also when running it for the first time on the command line using -vaz gives you more verbose output.
Topic revision: r4 - 23 Dec 2005, ChrisC
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback