IMC Mirror How To

The idea of this page is to link to all the documentation about how to set up mirrors of different IMC web sites.

See also IMCServerOffers and BackupHowTo and ApacheReverseProxy also ApacheLogsWithoutIPs for the log settings in httpd.conf.

The rsync server itself is being documented on the AhimsaRsync1Info page.

IMCs

Indymedia sites with rsync servers. All these sites run MiR - at the moment it is the only code base that generates static HTML (with SSIs).

Ambazonia

Public rsync server:

• rsync://rsync.ambazonia.indymedia.org/ambazonia.indymedia.org/

Armenia

Public rsync server:

• rsync://rsync.armenia.indymedia.org/armenia.indymedia.org/

Beirut

Public rsync server:

• rsync://rsync.beirut.indymedia.org/beirut.indymedia.org/

Bolivia

Public rsync server:

• rsync://rsync.bolivia.indymedia.org/bolivia.indymedia.org/

Brasil

Public rsync server:

• rsync://rsync.brasil.indymedia.org/brasil.indymedia.org/ | Brasil Mirror Howto

CH

Public rsync server:

• rsync://rsync.ch.indymedia.org/ch.indymedia.org/

Instructions: ChMirror

Ecuador

Public rsync server:

• rsync://rsync.ecuador.indymedia.org/ecuador.indymedia.org/

Euskal Herria

Public rsync server:

• rsync://rsync.euskalherria.indymedia.org/basque.indymedia.org/

Galiza

Public rsync server:

• rsync://rsync.galiza.indymedia.org/galiza.indymedia.org/

Germany

Public rsync server:

• rsync://ahimsa-rsync1.indymedia.org/germany.indymedia.org/

Milwaukee

Public rsync server:

• rsync://rsync.milwaukee.indymedia.org/milwaukee.indymedia.org/

New York

See the how-to at: NewYorkMirrors

Public rsync server:

• rsync://rsync.nyc.indymedia.org/nyc.indymedia.org/

Ottawa

Public rsync server:

• rsync://rsync.ottawa.indymedia.org/ottawa.indymedia.org/

Poland

Public rsync server:

• rsync://rsync.poland.indymedia.org/poland.indymedia.org/

Romania

Public rsync server:

• rsync://rsync.romania.indymedia.org/romania.indymedia.org/

San Diego

Public rsync server:

• rsync://rsync.sandiego.indymedia.org/sandiego.indymedia.org/

UK

Public rsync servers

primary rsync server

• rsync://rsync.indymedia.org.uk/www.indymedia.org.uk/ -- this is ahimsa-rsync1.indymedia.org

secondary rsync servers

• rsync://rsync1.indymedia.org.uk/www.indymedia.org.uk/ -- this is ahimsa-rsync1.indymedia.org

• rsync://rsync2.indymedia.org.uk/www.indymedia.org.uk/ -- this is rsync.nth-dimension.org.uk

• rsync://rsync3.indymedia.org.uk/www.indymedia.org.uk/ -- this is snake.indymedia.org

alt rsync server

• rsync://rsync.alt.indymedia.org.uk/www.indymedia.org.uk/ — this server is for people running mirrors on DSL / cable connections (please don't use the primary rsync server for this)

See also

See UkMirror for info about how the UK IMC mirrors work and how to set up a mirror of www.indymedia.org.uk.

US

Public rsync server:

• rsync://rsync.us.indymedia.org/us.indymedia.org/

WWW

See GlobalWebsitemirroring for the details, public rsync server:

• rsync://rsync.www.indymedia.org/www.indymedia.org/

Using rsync

• $ rsync rsync://ahimsa-rsync1.indymedia.org/
• www.indymedia.org.uk Indymedia UK rsync service.
• brasil.indymedia.org Indymedia Brasil rsync service.
• www.indymedia.org Indymedia WWW rsync service.
• us.indymedia.org Indymedia US rsync service.
• ch.indymedia.org Indymedia CH rsync service.
• poland.indymedia.org Indymedia Poland rsync service.
• sandiego.indymedia.org Indymedia San Diego rsync service.
• galiza.indymedia.org Indymedia Galiza rsync service.
• germany.indymedia.org Indymedia Germany rsync service.
• ambazonia.indymedia.org Indymedia Ambazonia rsync service.
• armenia.indymedia.org Indymedia Armenia rsync service.
• basque.indymedia.org Indymedia Basque rsync service.
• bolivia.indymedia.org Indymedia Bolivia rsync service.
• beirut.indymedia.org Indymedia Beirut rsync service.

Anyone can mirror any of the above sites using rsync. By running cron daily or so, you can maintain a nice backup of the site.

Here's an example command for mirroring bolivia that you can run on your GNU/Linux/*BSD box (in other words, it will download the /entire/ site to your local hard drive):

• rsync -avHP rsync://ahimsa-rsync1.indymedia.org/bolivia.indymedia.org/ .

Don't miss the trailing dot which represents the current working directory. You could also specify another directory name there if you want.

The amount of disk space for the sites:

• 154M ambazonia
• 2.0G basque
• 350M beirut
• 1.3G bolivia
• 17G brasil
• 2.8G ch
• 1.6G earth
• 449M ecuador
• 2.6G galiza
• 8.1G germany
• 1.9G poland
• 1.8G sandiego
• 9.4G uk
• 305M us

Some sites have their own DNS as well, for example UK is rsync.indymedia.org.uk.

Example rsync script

This script has been designed for the UK site but can be simply changed for another, the 3 variables you will probbaly want to change are SITE, RSYNC_SERVER and DESTINATION_BASE.

The advantage of using this script is that it uses a lock file so there won't more than one instance running unless unless it runs for longer than the timeout value (set to 2 hours below) and another instance does a forced removal of the lockfile. So... don't set up a cron job for updating your mirror untill you have completed an initial sync!

#!/bin/bash
# www.indymedia.org.uk mirror pull script

# this script uses lockfile which comes with procmail
# for file locking

# this script assumes the rsync base address is
# rsync://$RSYNC_SERVER/$SITE

# the site we are getting
SITE="www.indymedia.org.uk"
# the rsync server we are using
RSYNC_SERVER="rsync.indymedia.org.uk"

# set this to the location of the mirror on your filesystem
DESTINATION_BASE="/var/www/$SITE"

# check that we have write perms for the site
if [ ! -w "$DESTINATION_BASE" ]
  then
    echo "$DESTINATION_BASE needs to be writable by you."
  else

  # the user running this script needs to have 
  # write permissions for the lockfile
  LOCKFILE="$HOME/`basename $0`.$SITE.lock"

  # if the $LOCKFILE exists then exit
  # the lockfile is read only
  # the timeout is set to 2 hours (7200 secs)
  # if the lockfile is older than this it will be 
  # removed
  lockfile -r 1 -l 7200 $LOCKFILE || exit 23

  # check that the lock file is writable
  echo "Starting a sync for $SITE from $RSYNC_SERVER"; echo

  # our rsync command
  RSYNC="/usr/bin/rsync --timeout=360 -CLrptzvu"

  # lets do some stuff 
  ${RSYNC} rsync://${RSYNC_SERVER}/${SITE}/ ${DESTINATION_BASE}/

  # delete the lockfile
  rm -vf $LOCKFILE

fi