<< MiR

<< MirInstallation

You should probably also check out MiRInstallDebian - it is much shorter and more structured than the text below.

---

The 13 June 2005 (or later) installation scripts

• cvs -d :pserver:anonymous@cvs.codecoop.org:/cvsroot/mir co mir/scripts/mir-setup

are likely to be more up-to-date than this installation howto.

---

MIR INSTALLATION HOWTO

Last updated: $Date: 2005/08/22 20:48:16 $
----------------------------------------------------------------

Here is a short installation-howto of Mir, somewhat more explicit than
the earlier version.  Some Debian-specific instructions are given.  


prerequisites: 

- tomcat 4.0.4 (4.0.3 and below have some bad bugs) or above (3.3 works too as
  of 04.04.2002, but this could change)
  tomcat is available from http://jakarta.apache.org/tomcat/
- apache 1.3.x. with mod_jk.so. As far as I can tell the connector for 2.x is
  still rather undocumented. http://httpd.apache.org
- postgres 7.1.x or 7.2.x. http://www.postgresql.org
- ant (a java-based make) 
- jaxp-1.1 (a SAX 2.0 compliant XML parser, comes with ant >= 1.4)
- the JAI image framework (Java Advanced Imaging) versin 1.1.1 . get it from 
  java.sun.com. ** NOTE: because JAI uses a native acceration library (a .so)
  it must be placed in tomcat's "lib" (i.e $TOMCAT_HOME/common/lib) directory and
  not under the default webapps/Mir/WEB-INF/lib directory **
- A good reading of Tomcat, Apache and Postgresql documentation if you are not
  familiar with any of them. The documentation is available at:
  http://jakarta.apache.org/tomcat/tomcat-4.0-doc/index.html,
  http://httpd.apache.org/docs/ and http://www.postgresql.org respectively.
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
0. To set all this up on Debian:

make sure you have stable, testing, and unstable listings in
/etc/apt/sources.list.  THen do all the following as root or using sudo!

If you already have tomcat, and it's working well with apache, then
skip directly to (c).  If it's not working well, then make sure you
purge _all_ conf files & so forth ( though it never hurts to back them
up first).  "apt-get remove --purge" may not get rid of everything; try 

locate tomcat
and then delete all the files you find (again, if you're unsure what
they are, then back them up first).


(a) Install Java 
apt-get install j2sdk1.3 j2sdk1.3-doc

JAVA_HOME=/usr/lib/j2sdk1.3; export JAVA_HOME
PATH=$PATH:$JAVA_HOME/bin; export PATH
(add these lines to your .bashrc as well)

(b) Installing Tomcat and Apache

As root, do the following:
apt-get install apache 
 configure apache by editing httpd.conf:
* enable shtml includes:
  - add LoadModule includes_module /usr/lib/apache/1.3/mod_include.so
  - make sure your directory contains "Options Includes"
* Determine if you need to modify any apache mime-mappings
  - The web-server host must recognize the .m3u, .pls and other file extensions
    and send the proper "audio/x-mpegurl" and "audio/x-scpls" mime-types
    respectively.  If the web server is apache, it's easy, just
    add:
    
    audio/x-mpegurl                 m3u
    audio/x-scpl                    pls
    
    to the file pointed to by the "TypesConfig" command in your apache config
    file. Or add and equivalent AddType command to your httpd.conf.  Of course
    this assumes that the mod_mime is loaded.
* Make sure that if you are using a non standard character set enconding that 
  Apache doesn't accidentally send the wrong encoding in the HTTP headers.



Now deal with Tomcat:
apt-get install -t unstable tomcat4 tomcat4-webapps tomcat4-admin 
   --> this gets you tomcat4.1.  You can work with tomcat 4.0.4
   as well, but a couple of specific configuration elements are
   different.  To get 4.0.4, run the following:
   apt-get install -t testing tomcat4 tomcat4-webapps
apt-get install apachelib-java

Tomcat will be installed in /usr/share/tomcat4, so you should put 

CATALINA_HOME=/usr/share/tomcat4; export CATALINA_HOME
TOMCAT_HOME=$CATALINA_HOME; export TOMCAT_HOME

in your .bashrc; you might as well execute the ocmmand right now as
well.  

I found it somewhat difficult to get Tomcat and Apache working
   together, until I emailed the Debian package maintainer, who told
   me that he had purposely removed the mod_jk.conf-auto function from
   Tomcat4.1!  This is because Debian encourages Tomcat users to
   switch to mod_webapp instead.  You may want to try this option
   out.  I didn't, because I was able to get it working using the
   JkMount option.  So in /etc/apache/httpd.conf, look for a section
   like this at the end:  
                                   
   # The following line is for apacheconfig - DO NOT REMOVE!     
   JkWorkersFile /etc/tomcat/jk/workers.properties           
   Include /var/lib/tomcat/conf/mod_jk.conf           
                          
                           
   This has been insrted by libapache-java to try to get mod_jk to
   work. unfortunately it doesn't work with Tomcat 4.1, so replace it
   with the following:

                       
   # The following line is for apacheconfig - DO NOT REMOVE!     
   JkWorkersFile /etc/tomcat/jk/workers.properties           
   JkMount /*.jsp ajp13                    
   JkMount /Mir ajp13                    
   JkMount /Mir/* ajp13                    
   JkMount /servlet ajp13                    
   jkMount /examples/* ajp13                      
   #Include /var/lib/tomcat/conf/mod_jk.conf           
         

-> As of tomcat-4.1.30-6 under debian-sarge, you probably need the
JkWorkersFile line more like:
        JkWorkersFile /etc/libapache-mod-jk/workers.properties
                            
restart tomcat(as root):
/usr/share/tomcat4/bin/shutdown.sh
/usr/share/tomcat4/bin/startup.sh
restart apache(as root):
apachectl restart

In your browser, check to see if each pogram is working on its own:
http://localhost should give you the debian apache start page
http://localhost:8180 should give you the tomcat start page

now check to see if mod_jk is allowing the two programs to connect:
http://localhost:8180/examples/servlets/index.html should give you the
Tomcat example servlets.  

You can get apache to write the :8180 port for you with a Redirect,
e.g. in /etc/apache/httpd.conf put

Redirect /Mir http://:8180/Mir

where  is the full address of your host.


Once this is working, you should enable the tomcat manager
application:  
cd /usr/share/tomcat4/conf
open tomcat-users.xml in a text editor.  Add a line like this:



You should now be able to click on the "Manager" link on the Tomcat
home page, log in, and use the Manager functions.  This is extremely
convenient when you're eloading Mir multiple times later on!

(c) install postgres:
apt-get install postgresql postgresql-client postgresql-doc
--> the new postgres package should work fine with Mir.  Make sure you
enable UNICODE, and if asked about JDBC (you shouldn't be) make sure
to enable it as well. if you would like to try a graphical frontend
for the database, find one using

apt-cache show postgresql

fmailiarize yourself with the psql interface, and the function of the
"postgres" user.

(d) install JAI
go to:
http://java.sun.com/products/java-media/jai/downloads/download.html

and download the "CLASSPATH for Linux" version of the JAI package
(1.1.01 works fine, later versions have not been tested by me) to 
/usr/share/tomcat4/common/lib

now:

cd /usr/share/tomcat4/common/lib

gunzip "name_of_file.tar.gz" (don't use the quotes: replace with the
real name of the file you downloaded!)

tar -xvf "name_of_file.tar" 

Read the instructions in
/usr/share/tomcat4/common/lib/jai_whatever_version_you_have/INSTALL-jai.txt
but you should *ignore* the instruction to install the JRE and JDK binaries
of JAI - you *only* need the CLASSPATH files.

Now add the JAI files to your CLASSPATH and LSD_LIBRARY_PATH:  
JAIHOME=$TOMCAT_HOME/common/lib/jai-1_1_1_01/lib; export JAIHOME
CLASSPATH=$JAIHOME/jai_core.jar:$JAIHOME/jai_codec.jar:$JAIHOME/mlibwrapper_jai.jar:$CLASSPATH;
export CLASSPATH
LD_LIBRARY_PATH=.:$JAIHOME:$CLASSPATH; export LD_LIBRARY_PATH

--> make sure to replace "jai-1_1_1_01" in the above with the
version-number of your copy of JAI
--> as above, it's a good idea to add these lines to your .bashrc

---------------------------------------------------------------


(1) CONFIGURE MIR!

(a).  get the latest version:

cd to the directory where you would like to install the mir package

cd /absolute/path

CVS LOGIN:

  cvs -d :pserver:anonymous@mir.indymedia.org:/var/lib/cvs login
   password: anonymous

CVS CHECKOUT:

   cvs -d :pserver:anonymous@mir.indymedia.org:/var/lib/cvs co mir 


(b) customize the config: 

   cd mir/etc
   cp config.properties-dist config.properties 

now customize config.properties for your needs.  config.properties is
pretty well-documented, but read it carefully.  I had to recompile
several times before I got all the pathnames right.
 
Warning: ProductionHost, PublicationHost, OpenAction should be absolute
hostnames.

The following should be absolute URLs if they are on other hosts:

Producer.Video.Host=http:///video
Producer.Audio.Host=http:///audio
Producer.Image.Host=http:///images
Producer.Media.Host=http:///media/
Producer.RealMedia.Host=rtsp://some.streamingmedia.server/somedir/

but if they are local, then relative paths would be better, like
Producer.Image.Host=/images

and so on.

A trailing slash should not matter.


(c) configure the perms.sh file if neccessary -- IMPORTANT! READ THIS!
We provide a script that sets all files' and direcories' permissions to
a quite reasonable state. This script gets automagically called by
ant after compilationl. The most important thing you have to do after
compiling Mir is to ensure that the log files -- especially 
dbentity.log -- are not readable by users that could compromise 
system security, because all passwords and the like will be logged here.
   
   cp perms.sh-dist perms.sh

Now, change the install directory and group in perms.sh

   edit perms.sh 

-> This set of permissions does not always work. With one debian-sarge
installation, the permissions needed to be changed to
chown tomcat4:nogroup  instead of www-data:www-data




(d) Customize the templates
you need to edit the templates before compiling, or not all of your
changes will show up in the produced site.  If this is too
intimidating, don't worry -- you can always recompile!  It only takes
about a minute...

(e) check web.xml!
--> tomcat 4.1 has a different method of composing servlet URL's, so
if you are uasing 4.1, you wil need to make some minor changes to
mir/etc/web.xml



look for sections like this: 
  
        
            Mir
        
        
            /Mir
        
    

the url-pattern has to change; so replace the above with:
  
        
            Mir
        
        
            /servlet/Mir
        
    
make sure to do the same for OpenMir and OutputMir.

--> this comment may or may not be necessary as of Mir-20041106 CVS,
since the replacement section is already present in the web.xml file;
also, OutputMir is not present.


-------------------------------------------------------------
-------------------------------------------------------------
IMPORTANT!  It is a good idea to skip ahead to (2) and create the
Postgres database at this point.  Then cd back to the mir directory,
and continue.  
-------------------------------------------------------------
-------------------------------------------------------------

(f) compile. Make sure all the environment variables are set as
   indicated in section (0), or build.xml will not run properly:

su
ant

  This should take about a minute.  If you don't get an error, run
   perms.sh:
   sudo perms.sh


Do this as root so the permissions script is able to set
the permissions and owners correctly.
   

(g) Get Tomcat to recognize Mir.
Link in the webapps directory of tomcat to the install directory
(unless you changed the setting in config.properties, the 
directory is called "Mir" and is located in the same directory in which 
you installed the "mir" directory). (Here and in the rest of this document,
we assume you called the link "Mir", but this could be named anything.)
   cd /path/to/tomcat/webapps (tomcat-4.0.x/webapps)
   ln -s /path/to/Mir Mir

-> As of Mir-20041106 CVS, if you compile mir with ant in the directory
/path/to/Mir/
then  ant  will place the WEB-INF directory as  
/path/to/Mir/bin/mir/WEB-INF/

Then you should:
   cd /path/to/tomcat/webapps 
   ln -s /path/to/Mir/bin/mir/WEB-INF Mir 
where "Mir" is the same name as in 
 JkMount /Mir ajp13  
above 


dynamically reload Mir: 

http://localhost:8180/manager/stop?path=/Mir
if you're not using the Debian installation, you need to use:
http://localhost:8080/manager/stop?path=/Mir

if you're using Tomcat 3.3, you need to restart Tomcat (consult the
docs for how to do that).  


(h) Copy any dynamic library files ending with ".so" (so far only the JAI native
acceleration library found in the JAI package tarball or zip from sun) to your
$JAVA_HOME/jre/lib/i386 directory (where the other ".so" files live). Or, you
can skip the whole thing and live without "native" acceleration for image
manipulation.


(i) (this should probably be done at an earlier step)
-> As of as of Mir-20041106 CVS, you need to open up tomcat security,
at least in debian-sarge.
In the file 
  /etc/default/tomcat4
set
 TOMCAT4_SECURITY=no

You can probably add back in some sort of security. Please update
these INSTALL instructions and discuss this on the 
mir-coders at lists.indymedia.org mailing list.


(2) CREATE AND CONLFIGURE THE MIR DATABASE!

Review mir/etc/config.properties.  Look carefully at the entries you
put in the DATABASE SETUP section, especially the Username, Password,
Host, and Name variables.  If you don't know anything about Postgres,
look over the INSTALL.postgresql document, which is very helpful (but
remember, you don't need to install postgresql from scratch
anymore if you're on Debian woody!).  Please do not use the "postgres"
user in config.properties -- this is a serious security risk.  Choose
a username,a password, and a database name, then put those values in
config.properties if you haven't already.  

(a) Create the new database

We create and configure the database as posgresql user "postgres":

   createdb -U postgres --encoding=unicode Mir 
--> here and elsewhere, replace "Mir" with the name you already chose
for your database


(b) create an unprivileged database user for Mir
First, connect to the database as the database's superuser. 

   psql -U postgres Mir

Now we create the actual user. Please choose a password that is hard to 
guess instead of "joshua". Good passwords have characters and numerals in
it, have no link to its owner (like being her birthday, age, name of her 
husband, dog, child, car, favourite beer brand). A good password looks like
this: "8ncx4un".
    
    CREATE USER Mir WITH PASSWORD 'joshua' NOCREATEDB NOCREATEUSER;
--> again, don't just copy this into the psql command line -- replace
"Mir" and "joshua" with your username and password from config.properties

now exit: 
\q


(c) create base table
cd back to the mir home directory.  now execute the following:

   psql -Upostgres -f dbscripts/create_pg.sql Mir
now quit psql:
        \q
back in the shell, execute the dbscripts:
    for i in dbscripts/help*.sql ; do psql -Upostgres -f $i Mir ; done
    for i in dbscripts/populate*.sql ; do psql -Upostgres -f $i Mir ; done

(d) Grant the required permissions to the new user
First, make your new user into the database administrator for the new
database (instead of postgres):
psql -U postgres Mir
     select * from pg_database;
     select * from pg_user;
you'll see a display like this:
   usename    | usesysid | usecreatedb | usetrace | usesuper | usecatupd |  passwd  | valuntil
--------------+----------+-------------+----------+----------+-----------+----------+----------
 postgres     |        1 | t           | t        | t        | t         | ******** |
 matt         |      100 | t           | f        | t        | t         | ******** |
 mir          |      101 | f           | f        | f        | f         | ******** |

followed by

  datname  | datdba | encoding | datistemplate | datallowconn | datlastsysoid | datvacuumxid | datfrozenxid | datpath
-----------+--------+----------+---------------+--------------+---------------+--------------+--------------+---------

 template1 |      1 |        5 | t             | t            |         16554 |        11258 |   3221236731 |

 template0 |      1 |        5 | t             | f            |         16554 |           49 |           49 |

 Mir       |      1 |        5 | f             | t            |         16554 |        10805 |   3221236278 |
 
note the "usesysid" column for your new user, and the datname for your
new database.  now execute the following line, obviousjly using the
variable you just noted:

update pg_database set datdba=USESYSID_FROM_PG_USER where datname=DATABASENAME

now exit psql: 
\q

Save the following lines to a file called
set.permissions:

select 'grant all on '||relname||' to Mir;'
from pg_class
where relname not like 'pg%'
order by relname;

-->be sure to replace "Mir" with your username!!!

-> To a newbie user, it is not obvious how to execute this file.
One method that works is the following:

 (A) start up psql
 (B) type in the above four lines -> you should be given many lines of
output - cut/paste the output to a text file in another window - you
will be able to spacebar through the full list of output lines,
probably several screens long 
 (C) now cut/paste these output list of instructions back into the 
psql window

Then again use click and paste to paste all of these instructions into your
running psql shell.


(e) Apply neccessary changes to config.properties

--> if you folowed the instructions above, you shouldn't have to do
this, but doublecheck!
Please open config.properties and look for the lines that begin with
"Database.". The interesting properties are "Username", "Password", "Host"
and "Name". Change these properties so that they reflect the settings you
used to create the database and the user.

You should make sure that no copy of config.properties (neither in mir nor
in Mir/src nor in Mir/WEB-INF/classes nor in the directory tree you compiled
Mir from) is world-readable. Else you wouldn't have to install a password,
anyway.

(f) Setup PostgreSQL so that all connections have to pass a password

In /etc/postgresql/pg_hba.conf you should make sure that nobody can
use the database without a password, by inserting these lines:

local        all                  password
host         all         127.0.0.1     255.0.0.0           password
host         all         0.0.0.0       0.0.0.0             reject

make sure you comment out all other permissions lines in this file!

This means: All local connections (i.e. psql without "-h hostname" option)
have to authenticate themselves with a password. All connections from
localhost (127.0.0.1) have to supply a password, too. All other connections
are rejected. This line doen't have to be there if you have a properly
configured firewall but even if you do have one, it adds to the security in
case an attacker penetrates the firewall by some hack.

If you can't access PostgreSQL after this for any reason, try and change
"password" in /etc/postgresql/pg_hba.conf to "trust". This should disable
any authentication method and make the database accessible again. Please use 
this setting only temporarily because anybody who can access the PostgreSQL
server could take over the database completely this way. After you fixed
your password setting, switch the setting back to "password".
You may want to change your PostgreSQL password from time to time to make
database takeover harder. Rememer: Security is a process.


(g) If you are installing a new mir setup with an old database,
then you can put your old database into your new site with something
like:

pg_restore -d  -U  -W 


----------------------------------------------------------------

AT THIS POINT YOU SHOULD HAVE A WORKING INSTAATION OF MIR!  CHECK TO
MAKE SURE -- DON'T BOTHER WITH THE REST TILL YOIU'VE MADE MIR WORK ONCE!


Goodies:

(3) Add the dupe prevention trigger to the database:
   cd mir/dbscripts/dupetrigger
   
   There, read INSTALL and follow the instructions.
   

(4) Tweak mime-type extensions mappings in etc/web.xml file.

*** Note the defaults should be o.k for most installations ***

Add or remove any mime types you wish to support. This is used to figure
out the mime-type when (broken browsers?) browsers don't send the mime-type
in the content-type header field when uploading a media file. Note at the
moment you still have to add these to the media_type SQL table as well which
maps the mime-types to the correct mediaHandler class. See the comments in
the MirMedia class in javadoc for more details.

11. restart tomcat 

13. configure apache

edit http.conf:
* set the document root to the same directory as in the mir config file
* enable shtml includes:
  - add LoadModule includes_module /usr/lib/apache/1.3/mod_include.so
  - make sure your directory contains "Options Includes"
* Determine if you need to modify any apache mime-mappings
  - The web-server host must recognize the .m3u, .pls and other file extensions
    and send the proper "audio/x-mpegurl" and "audio/x-scpls" mime-types
    respectively.  If the web server is apache, it's easy, just
    add:
    
    audio/x-mpegurl                 m3u
    audio/x-scpl                    pls
    
    to the file pointed to by the "TypesConfig" command in your apache config
    file. Or add an equivalent AddType command to your httpd.conf.  Of course
    this assumes that the mod_mime is loaded.

that's it :)

now the admin-application is accesable via:  

   http://host/Mir/servlet/Mir 

and the openposting-servlet via  
   
   http://host/Mir/servlet/OpenMir

standard login is admin/indymedia. See the webdb_users SQL table to change/add
users or passwords.


14. cities.inc

You can add a cronjob as the user who has write access to your mir
installation

crontab -e

For a daily update add a line like:

04 22 * * * /path/to/cities.sh >/dev/null 2>&1

where you create a file like

/path/to/private/scripts/cities.sh

containing

#!/bin/sh
wget http://www.indymedia.org/cities.inc
cp -p cities.inc /path/to/Mir/bin/mir/WEB-INF/etc/producer/cities.inc

where /path/to/Mir/bin/mir/WEB-INF is the place where your compiled
mir files are found (see above: ln -s soft link from tomcat/webapps).

You can use another application like curl if you don't have wget.


15. automatic redirection to your preferred index page

If you want the user to automatically go to a welcome page, then it
seems you need to create a file like    index.php  in the root 
directory of the site containing something like:



where pl/index.shtml is the welcome page.


16. images, media, icon directories 

In config.properties, you probably have something like:

Producer.Image.Host=/images
Producer.Media.Host=/media
Producer.Image.Path=//site/images/
Producer.Media.Path=//site/media
Producer.Image.IconPath=/icon

You need to make sure that the directories images/, media/ and icon/ 
exist and have the right protections, e.g. in / do

mkdir images media icon
chown www-data:nogroup images media icon  

where the chown command has the options you used running perms.sh above.

If you are installing a new mir setup with an old database, you need to 
copy the old images/ and media/ files to your new setup.


17. Syndication (e.g. to www.indymedia.org)

Make sure that you have rss and js template files in the etc/producer
directory.


SEARCHING

The Mir code offers no internal search facilities, rather, the design
expects the use of an external program to crawl and index the static
site.  One (recommended) tool for doing this is htdig
(http://htdig.org), which generates static databases of the site
content and then accesses those databases through a very fast CGI
program written in C.  In the scripts directory, a perl CGI script 
which wraps calls to htsearch is provided (scripts/search.pl) which
will allow searching based off of media type.  (This is possible
because the standard templates will include META keywords like
hasAudio, hasVideo, etc.)  

UPGRADING

see the UPGRADING.mir file.

TROUBLESHOOTING

You can give these a try if anything goes wrong:

+ Restart Tomcat. Especially after compiling the sources Tomcat has to be
  restarted.

+ Check file permissions and ownership. Try and run perms.sh.

* Look through 
/path/to/Mir/bin/mir/WEB-INF/log/*.log  files for hints,
e.g. 
tail -100 /path/to/Mir/bin/mir/WEB-INF/log/producer.log | more


----------------------------------------------------------------

$Date: 2005/08/22 20:48:16 $ - the Mir coders