FreeBSD^?' jails

There are jail(8)s setup in Berkman (see BerkmanInfo), here some words about them.

You'll probably figure out things checking out the man page, but here are a couple of non-really tech words about how jails work and how they feel.

Basic concept

You use a jail to imprison a process and all its descendants, it really looks like chroot(8), except that there are a number of additional enforcements and options: basically you it is assumed that you can give people you don't really trust root access in a jail of their own.

A jail has an IP address configured. This address can be the same as the one of the host or different (an alias). Assuming that host has IP A, B and C and there are jails listening A and B, the ports that aren't listen(3)ed by the host fallback to the jails. These jails can't manage to open a port on C (unless there are bugs in the jail subsystem, of course).

So: on berkman, if jail-a has IP berkman.i.o and Apache isn't running on the main host, connections to port 80 will be catched by the jailed process.

Same for SSHD. If you have only one IP address, it is better to have jailed SSHDs listening on other ports than 22 (23, 24, say).

You can have jails listening non-routeable addresses such as 127/8. You can put your "private" services there, databases, etc. If you change your mind and actually want to offer your Postgres to the World, you'll just need to restart your jail with a routeable IP address.

Remarks & hints

Jails do not need to be complete systems, they can be made very chroot like, a la "chroot /var/named named", except for the hostname and ip params one needs to give for jails. You can safely delete programs and file from the jail - if the file was important, you still can copy it from the base system.

PS is weird

You can send signals to a process in a jail from the host system: no need to log in to kill -HUP apache, for example; also running a 'ps' shows what process are jailed - their status field ends with a 'J':

$ ps ax | grep J
     624  ??  SsJ    0:00.11 /usr/sbin/syslogd -s
     636  ??  IsJ    0:00.60 /usr/sbin/named -u bind -g bind
     770  ??  IsJ    0:00.11 /usr/sbin/sshd
     789  ??  IsJ    0:00.17 /usr/sbin/cron
     991  ??  SsJ    0:00.13 /usr/sbin/syslogd -s
    1153  ??  IsJ    0:00.17 /usr/sbin/cron

From the base system, "ps" resolves UID with its own /etc/password. Which means that if you've user foo with UID 1003 in the host and user bar with UID 1003 in a jail, "ps" will report nothing about bar and will wrongly attribute to foo its process. This is disturbing, in general you want to have users with the same UID in the host and in jails, even if they have /sbin/nologin as a shell in the host.

DF is weird

A number of stats are weird, when seen from a jail, those given by "df", as an example, from 'jail-local':

   jail-local# df
   Filesystem  1K-blocks     Used     Avail Capacity Mounted on
   /dev/da0s1a   1007950    46988    880326     5%   /
   /dev/da0s1e  10080382  4790602   4483350    52%   /usr
   /dev/da0s1f   5161198   599410   4148894    13%   /var
   /dev/da0s1g   5161198  1778384   2969920    37%   /tmp
   procfs              4        4         0   100%   /proc
   /dev/da0s1h 325639946 63071828 236516924    21%   /home
   procfs              4        4         0   100%   /home/JAILS/jail-a/proc
   procfs              4        4         0   100%   /home/JAILS/jail-b/proc
   procfs              4        4         0   100%   /home/JAILS/jail-local/proc

The /usr here is the one of the base system, oh well.

TOP doesn't work

   jail-a# top
   kvm_open: short read
   top: kvm_read for _ccpu: Unknown error: 0
   kinda problematic for monitoring performance

Indeed.

More information

Checkout jail(8), the manpage for jail on the FreeBSD site.

Warning

The problem when you have jails is that it quits you the stress of modifying extremely important config files like "rc.conf" and do dangerous operations. It's likely that someone will happily mess the host "rc.conf" or shoot a "kill -9 -1" or something while actually being running in the host. I wonder who that will be.

-- JiBe - 10 Feb 2004 - blah blah.

to top