This section is based upon an idea originally presented by Simon L. Nielsen <[email protected]>
at https://simon.nitro.dk/service-jails.html, and an updated article
written by Ken Tom <[email protected]>. This section illustrates
how to set up a FreeBSD system that adds an additional layer of security, using the jail(8) feature. It is
also assumed that the given system is at least running RELENG_6_0 and the information
provided earlier in this chapter has been well understood.
One of the major problems with jails is the management of their upgrade process. This
tends to be a problem because every jail has to be rebuilt from scratch whenever it is
updated. This is usually not a problem for a single jail, since the update process is
fairly simple, but can be quite time consuming and tedious if a lot of jails are
created.
Warning: This setup requires advanced experience with FreeBSD and usage of its
features. If the presented steps below look too complicated, it is advised to take a look
at a simpler system such as sysutils/ezjail, which provides an easier method of
administering FreeBSD jails and is not as sophisticated as this setup.
This idea has been presented to resolve such issues by sharing as much as is possible
between jails, in a safe way -- using read-only mount_nullfs(8)
mounts, so that updating will be be simpler, and putting single services into individual
jails will become more attractive. Additionally, it provides a simple way to add or
remove jails as well as a way to upgrade them.
Note: Examples of services in this context are: an HTTP server, a DNS server, a
SMTP server, and so forth.
The goals of the setup described in this section are:
Create a simple and easy to understand jail structure. This implies not having to run a full installworld on
each and every jail.
Make it easy to add new jails or remove existing ones.
Make it easy to update or upgrade existing jails.
Make it possible to run a customized FreeBSD branch.
Be paranoid about security, reducing as much as possible the possibility of
compromise.
Save space and inodes, as much as possible.
As it has been already mentioned, this design relies heavily on having a single master
template which is read-only (known as nullfs) mounted into
each jail and one read-write device per jail. A device can be a separate physical disc, a
partition, or a vnode backed md(4) device. In this
example, we will use read-write nullfs mounts.
The file system layout is described in the following list:
Each jail will be mounted under the /home/j directory.
/home/j/mroot is the template for each jail and the
read-only partition for all of the jails.
A blank directory will be created for each jail under the /home/j directory.
Each jail will have a /s directory, that will be linked to
the read-write portion of the system.
Each jail shall have its own read-write system that is based upon /home/j/skel.
Each jailspace (read-write portion of each jail) shall be created in /home/js.
Note: This assumes that the jails are based under the /home partition. This can, of course, be changed to anything else,
but this change will have to be reflected in each of the examples below.
This section will describe the steps needed to create the master template that will be
the read-only portion for the jails to use.
It is always a good idea to update the FreeBSD system to the latest -RELEASE branch.
Check the corresponding Handbook Chapter to accomplish this task. In the case the update is not
feasible, the buildworld will be required in order to be able to proceed. Additionally,
the sysutils/cpdup package will be required. We will use the portsnap(8) utility to
download the FreeBSD Ports Collection. The Handbook Portsnap Chapter is always good reading for newcomers.
First, create a directory structure for the read-only file system which will contain
the FreeBSD binaries for our jails, then change directory to the FreeBSD source tree and
install the read-only file system to the jail template:
Use mergemaster to install missing configuration files.
Then get rid of the extra directories that mergemaster
creates:
#mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i#cd /home/j/skel#rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev
Now, symlink the read-write file system to the read-only file system. Please make sure
that the symlinks are created in the correct s/ locations. Real
directories or the creation of directories in the wrong locations will cause the
installation to fail.
As a last step, create a generic /home/j/skel/etc/make.conf
with its contents as shown below:
WRKDIRPREFIX?= /s/portbuild
Having WRKDIRPREFIX set up this way will make it possible to
compile FreeBSD ports inside each jail. Remember that the ports directory is part of the
read-only system. The custom path for WRKDIRPREFIX allows builds
to be done in the read-write portion of every jail.
Now that we have a complete FreeBSD jail template, we can setup and configure the
jails in /etc/rc.conf. This example demonstrates the creation
of 3 jails: “NS”, “MAIL” and “WWW”.
Put the following lines into the /etc/fstab file, so that
the read-only template for the jails and the read-write space will be available in the
respective jails:
Note: Partitions marked with a 0 pass number are not checked by fsck(8) during boot,
and partitions marked with a 0 dump number are not backed up by dump(8). We do not
want fsck to check nullfs mounts or
dump to back up the read-only nullfs mounts of the jails. This
is why they are marked with “0 0” in the last two columns of each fstab entry above.
Warning: The reason why the jail_name_rootdir variable is set to /usr/home instead of /home is that the
physical path of the /home directory on a default FreeBSD
installation is /usr/home. The jail_name_rootdir variable must not be set to a path which includes a
symbolic link, otherwise the jails will refuse to start. Use the realpath(1) utility to
determine a value which should be set to this variable. Please see the
FreeBSD-SA-07:01.jail Security Advisory for more information.
Create the required mount points for the read-only file system of each jail:
#mkdir /home/j/ns /home/j/mail /home/j/www
Install the read-write template into each jail. Note the use of sysutils/cpdup, which helps to ensure that a correct copy is
done of each directory:
In this phase, the jails are built and prepared to run. First, mount the required file
systems for each jail, and then start them using the /etc/rc.d/jail script:
#mount -a#/etc/rc.d/jail start
The jails should be running now. To check if they have started correctly, use the jls(8) command. Its
output should be similar to the following:
At this point, it should be possible to log onto each jail, add new users or configure
daemons. The JID column indicates the jail identification number
of each running jail. Use the following command in order to perform administrative tasks
in the jail whose JID is 3:
In time, there will be a need to upgrade the system to a newer version of FreeBSD,
either because of a security issue, or because new features have been implemented which
are useful for the existing jails. The design of this setup provides an easy way to
upgrade existing jails. Additionally, it minimizes their downtime, as the jails will be
brought down only in the very last minute. Also, it provides a way to roll back to the
older versions should any problems occur.
The first step is to upgrade the host system in the usual manner. Then create a new
temporary read-only template in /home/j/mroot2.
#mkdir /home/j/mroot2#cd /usr/src#make installworld DESTDIR=/home/j/mroot2#cd /home/j/mroot2#cpdup /usr/src usr/src#mkdir s
The installworld run creates a few unnecessary
directories, which should be removed:
#chflags -R 0 var#rm -R etc var root usr/local tmp
Recreate the read-write symlinks for the master file system:
Note: The read-write systems are attached to the read-only system (/s) and must be unmounted first.
Move the old read-only file system and replace it with the new one. This will serve as
a backup and archive of the old read-only file system should something go wrong. The
naming convention used here corresponds to when a new read-only file system has been
created. Move the original FreeBSD Ports Collection over to the new file system to save
some space and inodes:
At this point the new read-only template is ready, so the only remaining task is to
remount the file systems and start the jails:
#mount -a#/etc/rc.d/jail start
Use jls(8) to check if the
jails started correctly. Do not forget to run mergemaster in each jail. The configuration
files will need to be updated as well as the rc.d scripts.