Once you have synchronized your local source tree against a particular version of
FreeBSD (FreeBSD-STABLE, FreeBSD-CURRENT, and so on) you can then use the source tree to
rebuild the system.
Make a Backup: It cannot be stressed enough how important it is to make a
backup of your system before you do
this. While rebuilding the world is (as long as you follow these instructions) an easy
task to do, there will inevitably be times when you make mistakes, or when mistakes made
by others in the source tree render your system unbootable.
Make sure you have taken a backup. And have a fixit floppy or bootable CD at hand. You
will probably never have to use it, but it is better to be safe than sorry!
Subscribe to the Right Mailing List: The FreeBSD-STABLE and FreeBSD-CURRENT
branches are, by their nature, in
development. People that contribute to FreeBSD are human, and mistakes
occasionally happen.
Sometimes these mistakes can be quite harmless, just causing your system to print a
new diagnostic warning. Or the change may be catastrophic, and render your system
unbootable or destroy your file systems (or worse).
If problems like these occur, a “heads up” is posted to the appropriate
mailing list, explaining the nature of the problem and which systems it affects. And an
“all clear” announcement is posted when the problem has been solved.
Do not use make world: A lot of older documentation
recommends using make world for this. Doing that skips some
important steps and should only be used if you are sure of what you are doing. For almost
all circumstances make world is the wrong thing to do, and the
procedure described here should be used instead.
To update your system, you should check /usr/src/UPDATING
for any pre-buildworld steps necessary for your version of the sources and then use the
following procedure:
Note: There are a few rare cases when an extra run of mergemaster -p is needed before the buildworld step. These are described in UPDATING. In general, though, you can safely omit this step if you
are not updating across one or more major FreeBSD versions.
After installkernel finishes successfully, you should boot
in single user mode (i.e. using boot -s from the loader
prompt). Then run:
Read Further Explanations: The sequence described above is only a short resume
to help you getting started. You should however read the following sections to clearly
understand each step, especially if you want to use a custom kernel configuration.
Before you do anything else, read /usr/src/UPDATING (or the
equivalent file wherever you have a copy of the source code). This file should contain
important information about problems you might encounter, or specify the order in which
you might have to run certain commands. If UPDATING contradicts
something you read here, UPDATING takes precedence.
Important: Reading UPDATING is not an acceptable
substitute for subscribing to the correct mailing list, as described previously. The two
requirements are complementary, not exclusive.
Examine the files /usr/share/examples/etc/make.conf and /etc/make.conf. The first contains some default defines - most of
which are commented out. To make use of them when you rebuild your system from source,
add them to /etc/make.conf. Keep in mind that anything you add
to /etc/make.conf is also used every time you run make, so it is a good idea to set them to something sensible for
your system.
A typical user will probably want to copy the CFLAGS and NO_PROFILE lines found in /usr/share/examples/etc/make.conf to /etc/make.conf and uncomment them.
Examine the other definitions (COPTFLAGS, NOPORTDOCS and so on) and decide if they are relevant to you.
The /etc directory contains a large part of your system's
configuration information, as well as scripts that are run at system startup. Some of
these scripts change from version to version of FreeBSD.
Some of the configuration files are also used in the day to day running of the system.
In particular, /etc/group.
There have been occasions when the installation part of make
installworld has expected certain usernames or groups to exist. When performing an
upgrade it is likely that these users or groups did not exist. This caused problems when
upgrading. In some cases make buildworld will check to see if
these users or groups exist.
An example of this is when the smmsp user was added. Users
had the installation process fail for them when mtree(8) was trying to
create /var/spool/clientmqueue.
The solution is to run mergemaster(8) in
pre-buildworld mode by providing the -p option. This will
compare only those files that are essential for the success of buildworld or installworld. If your
old version of mergemaster does not support -p, use the new version in the source tree when running for the
first time:
You may want to compile the system in single user mode. Apart from the obvious benefit
of making things go slightly faster, reinstalling the system will touch a lot of
important system files, all the standard system binaries, libraries, include files and so
on. Changing these on a running system (particularly if you have active users on the
system at the time) is asking for trouble.
Another method is to compile the system in multi-user mode, and then drop into single
user mode for the installation. If you would like to do it this way, simply hold off on
the following steps until the build has completed. You can postpone dropping to single
user mode until you have to installkernel or installworld.
As the superuser, you can execute:
#shutdown now
from a running system, which will drop it to single user mode.
Alternatively, reboot the system, and at the boot prompt, select the “single
user” option. The system will then boot single user. At the shell prompt you should
then run:
#fsck -p#mount -u /#mount -a -t ufs#swapon -a
This checks the file systems, remounts / read/write, mounts
all the other UFS file systems referenced in /etc/fstab and
then turns swapping on.
Note: If your CMOS clock is set to local time and not to GMT (this is true if
the output of the date(1) command does
not show the correct time and zone), you may also need to run the following command:
#adjkerntz -i
This will make sure that your local time-zone settings get set up correctly -- without
this, you may later run into some problems.
It is a good idea to save the output you get from running make(1) to another
file. If something goes wrong you will have a copy of the error message. While this might
not help you in diagnosing what has gone wrong, it can help others if you post your
problem to one of the FreeBSD mailing lists.
The easiest way to do this is to use the script(1) command,
with a parameter that specifies the name of the file to save all output to. You would do
this immediately before rebuilding the world, and then type exit when the process has finished.
If you do this, do not save the
output in /tmp. This directory may be cleared next time you
reboot. A better place to store it is in /var/tmp (as in the
previous example) or in root's home directory.
(unless, of course, your source code is elsewhere, in which case change to that
directory instead).
To rebuild the world you use the make(1) command. This
command reads instructions from the Makefile, which describes
how the programs that comprise FreeBSD should be rebuilt, the order in which they should
be built, and so on.
The general format of the command line you will type is as follows:
#make -x -DVARIABLEtarget
In this example, -x is
an option that you would pass to make(1). See the make(1) manual page
for an example of the options you can pass.
-DVARIABLE passes a
variable to the Makefile. The behavior of the Makefile is controlled by these variables. These are the same
variables as are set in /etc/make.conf, and this provides
another way of setting them.
#make -DNO_PROFILE target
is another way of specifying that profiled libraries should not be built, and
corresponds with the
target tells make(1) what you want
to do. Each Makefile defines a number of different
“targets”, and your choice of target determines what happens.
Some targets are listed in the Makefile, but are not meant
for you to run. Instead, they are used by the build process to break out the steps
necessary to rebuild the system into a number of sub-steps.
Most of the time you will not need to pass any parameters to make(1), and so your
command like will look like this:
#make target
Where target will be one of many build options.
The first target should always be buildworld.
As the names imply, buildworld builds a complete new tree
under /usr/obj, and installworld,
another target, installs this tree on the current machine.
Having separate options is very useful for two reasons. First, it allows you to do the
build safe in the knowledge that no components of your running system will be affected.
The build is “self hosted”. Because of this, you can safely run buildworld on a machine running in multi-user mode with no fear
of ill-effects. It is still recommended that you run the installworld part in single user mode, though.
Secondly, it allows you to use NFS mounts to upgrade multiple machines on your
network. If you have three machines, A, B
and C that you want to upgrade, run make
buildworld and make installworld on A. B and C should
then NFS mount /usr/src and /usr/obj
from A, and you can then run make
installworld to install the results of the build on B and
C.
Although the world target still exists, you are strongly
encouraged not to use it.
Run
#make buildworld
It is possible to specify a -j option to make which will cause it to spawn several simultaneous processes.
This is most useful on multi-CPU machines. However, since much of the compiling process
is IO bound rather than CPU bound it is also useful on single CPU machines.
On a typical single-CPU machine you would run:
#make -j4 buildworld
make(1) will then have
up to 4 processes running at any one time. Empirical evidence posted to the mailing lists
shows this generally gives the best performance benefit.
If you have a multi-CPU machine and you are using an SMP configured kernel try values
between 6 and 10 and see how they speed things up.
Many factors influence the build time, but fairly recent machines may only take a one
or two hours to build the FreeBSD-STABLE tree, with no tricks or shortcuts used during
the process. A FreeBSD-CURRENT tree will take somewhat longer.
To take full advantage of your new system you should recompile the kernel. This is
practically a necessity, as certain memory structures may have changed, and programs like
ps(1) and top(1) will fail to
work until the kernel and source code versions are the same.
The simplest, safest way to do this is to build and install a kernel based on GENERIC. While GENERIC may not have all
the necessary devices for your system, it should contain everything necessary to boot
your system back to single user mode. This is a good test that the new system works
properly. After booting from GENERIC and verifying that your
system works you can then build a new kernel based on your normal kernel configuration
file.
On FreeBSD it is important to build world
before building a new kernel.
Note: If you want to build a custom kernel, and already have a configuration
file, just use KERNCONF=MYKERNEL like this:
Note that if you have raised kern.securelevel above 1 and you have set either the noschg or similar flags to your kernel binary, you might find it
necessary to drop into single user mode to use installkernel.
Otherwise you should be able to run both these commands from multi user mode without
problems. See init(8) for details
about kern.securelevel and chflags(1) for details
about the various file flags.
If you were building a version of FreeBSD recent enough to have used make buildworld then you should now use installworld to install the new system binaries.
Run
#cd /usr/src#make installworld
Note: If you specified variables on the make
buildworld command line, you must specify the same variables in the make installworld command line. This does not necessarily hold true
for other options; for example, -j must never be used with
installworld.
For example, if you ran:
#make -DNO_PROFILE buildworld
you must install the results with:
#make -DNO_PROFILE installworld
otherwise it would try to install profiled libraries that had not been built during
the make buildworld phase.
Remaking the world will not update certain directories (in particular, /etc, /var and /usr) with new or changed configuration files.
The simplest way to update these files is to use mergemaster(8), though
it is possible to do it manually if you would prefer to do that. Regardless of which way
you choose, be sure to make a backup of /etc in case anything
goes wrong.
The mergemaster(8) utility
is a Bourne script that will aid you in determining the differences between your
configuration files in /etc, and the configuration files in the
source tree /usr/src/etc. This is the recommended solution for
keeping the system configuration files up to date with those located in the source
tree.
To begin simply type mergemaster at your prompt, and watch it
start going. mergemaster will then build a temporary root
environment, from / down, and populate it with various system
configuration files. Those files are then compared to the ones currently installed in
your system. At this point, files that differ will be shown in diff(1) format, with
the + sign representing added or modified lines, and - representing lines that will be either removed completely, or
replaced with a new line. See the diff(1) manual page
for more information about the diff(1) syntax and how
file differences are shown.
mergemaster(8) will
then show you each file that displays variances, and at this point you will have the
option of either deleting the new file (referred to as the temporary file), installing
the temporary file in its unmodified state, merging the temporary file with the currently
installed file, or viewing the diff(1) results
again.
Choosing to delete the temporary file will tell mergemaster(8) that we
wish to keep our current file unchanged, and to delete the new version. This option is
not recommended, unless you see no reason to change the current file. You can get help at
any time by typing ? at the mergemaster(8) prompt.
If the user chooses to skip a file, it will be presented again after all other files have
been dealt with.
Choosing to install the unmodified temporary file will replace the current file with
the new one. For most unmodified files, this is the best option.
Choosing to merge the file will present you with a text editor, and the contents of
both files. You can now merge them by reviewing both files side by side on the screen,
and choosing parts from both to create a finished product. When the files are compared
side by side, the l key will select the left contents and the r key will select contents from your right. The final output will be a
file consisting of both parts, which can then be installed. This option is customarily
used for files where settings have been modified by the user.
Choosing to view the diff(1) results again
will show you the file differences just like mergemaster(8) did
before prompting you for an option.
After mergemaster(8) is done
with the system files you will be prompted for other options. mergemaster(8) may ask
if you want to rebuild the password file and will finish up with an option to remove
left-over temporary files.
If you wish to do the update manually, however, you cannot just copy over the files
from /usr/src/etc to /etc and have it
work. Some of these files must be “installed” first. This is because the /usr/src/etc directory is not a copy of what your /etc
directory should look like. In addition, there are files that should be in /etc that are not in /usr/src/etc.
The simplest way to do this by hand is to install the files into a new directory, and
then work through them looking for differences.
Backup Your Existing /etc: Although, in theory,
nothing is going to touch this directory automatically, it is always better to be sure.
So copy your existing /etc directory somewhere safe. Something
like:
#cp -Rp /etc /etc.old
-R does a recursive copy, -p
preserves times, ownerships on files and suchlike.
You need to build a dummy set of directories to install the new /etc and other files into. /var/tmp/root
is a reasonable choice, and there are a number of subdirectories required under this as
well.
#mkdir /var/tmp/root#cd /usr/src/etc#make DESTDIR=/var/tmp/root distrib-dirs distribution
This will build the necessary directory structure and install the files. A lot of the
subdirectories that have been created under /var/tmp/root are
empty and should be deleted. The simplest way to do this is to:
#cd /var/tmp/root#find -d . -type d | xargs rmdir 2>/dev/null
This will remove all empty directories. (Standard error is redirected to /dev/null to prevent the warnings about the directories that are
not empty.)
/var/tmp/root now contains all the files that should be
placed in appropriate locations below /. You now have to go
through each of these files, determining how they differ with your existing files.
Note that some of the files that will have been installed in /var/tmp/root have a leading “.”. At the time of
writing the only files like this are shell startup files in /var/tmp/root/ and /var/tmp/root/root/,
although there may be others (depending on when you are reading this). Make sure you use
ls -a to catch them.
The simplest way to do this is to use diff(1) to compare the
two files:
#diff /etc/shells /var/tmp/root/etc/shells
This will show you the differences between your /etc/shells
file and the new /var/tmp/root/etc/shells file. Use these to
decide whether to merge in changes that you have made or whether to copy over your old
file.
Name the New Root Directory (/var/tmp/root) with a Time
Stamp, so You Can Easily Compare Differences Between Versions: Frequently rebuilding
the world means that you have to update /etc frequently as
well, which can be a bit of a chore.
You can speed this process up by keeping a copy of the last set of changed files that
you merged into /etc. The following procedure gives one idea of
how to do this.
Make the world as normal. When you want to update /etc and
the other directories, give the target directory a name based on the current date. If you
were doing this on the 14th of February 1998 you could do the following:
#mkdir /var/tmp/root-19980214#cd /usr/src/etc#make DESTDIR=/var/tmp/root-19980214 \
distrib-dirs distribution
Merge in the changes from this directory as outlined above.
Do not remove the /var/tmp/root-19980214 directory when you have finished.
When you have downloaded the latest version of the source and remade it, follow step
1. This will give you a new directory, which might be called /var/tmp/root-19980221 (if you wait a week between doing
updates).
You can now see the differences that have been made in the intervening week using diff(1) to create a
recursive diff between the two directories:
#cd /var/tmp#diff -r root-19980214 root-19980221
Typically, this will be a much smaller set of differences than those between /var/tmp/root-19980221/etc and /etc.
Because the set of differences is smaller, it is easier to migrate those changes across
into your /etc directory.
You can now remove the older of the two /var/tmp/root-*
directories:
#rm -rf /var/tmp/root-19980214
Repeat this process every time you need to merge in changes to /etc.
You can use date(1) to automate
the generation of the directory names:
You are now done. After you have verified that everything appears to be in the right
place you can reboot the system. A simple shutdown(8) should do
it:
You should now have successfully upgraded your FreeBSD system. Congratulations.
If things went slightly wrong, it is easy to rebuild a particular piece of the system.
For example, if you accidentally deleted /etc/magic as part of
the upgrade or merge of /etc, the file(1) command will
stop working. In this case, the fix would be to run:
23.4.14.1. Do I need to re-make the
world for every change?
There is no easy answer to this one, as it depends on the nature of the change.
For example, if you just ran CVSup, and it has shown the
following files as being updated:
it probably is not worth rebuilding the entire world. You could just go to the
appropriate sub-directories and make all install, and that's
about it. But if something major changed, for example src/lib/libc/stdlib then you should either re-make the world, or at
least those parts of it that are statically linked (as well as anything else you might
have added that is statically linked).
At the end of the day, it is your call. You might be happy re-making the world every
fortnight say, and let changes accumulate over that fortnight. Or you might want to
re-make just those things that have changed, and be confident you can spot all the
dependencies.
And, of course, this all depends on how often you want to upgrade, and whether you are
tracking FreeBSD-STABLE or FreeBSD-CURRENT.
23.4.14.2. My compile failed with
lots of signal 11 (or other signal number) errors. What has happened?
This is normally indicative of hardware problems. (Re)making the world is an
effective way to stress test your hardware, and will frequently throw up memory problems.
These normally manifest themselves as the compiler mysteriously dying on receipt of
strange signals.
A sure indicator of this is if you can restart the make and it dies at a different
point in the process.
In this instance there is little you can do except start swapping around the
components in your machine to determine which one is failing.
23.4.14.3. Can I remove /usr/obj when I have finished?
The short answer is yes.
/usr/obj contains all the object files that were produced
during the compilation phase. Normally, one of the first steps in the make buildworld process is to remove this directory and start
afresh. In this case, keeping /usr/obj around after you have
finished makes little sense, and will free up a large chunk of disk space (currently
about 340 MB).
However, if you know what you are doing you can have make
buildworld skip this step. This will make subsequent builds run much faster, since
most of sources will not need to be recompiled. The flip side of this is that subtle
dependency problems can creep in, causing your build to fail in odd ways. This frequently
generates noise on the FreeBSD mailing lists, when one person complains that their build
has failed, not realizing that it is because they have tried to cut corners.
23.4.14.4. Can interrupted builds be
resumed?
This depends on how far through the process you got before you found a
problem.
In general (and this is not a
hard and fast rule) the make buildworld process builds new
copies of essential tools (such as gcc(1), and make(1)) and the
system libraries. These tools and libraries are then installed. The new tools and
libraries are then used to rebuild themselves, and are installed again. The entire system
(now including regular user programs, such as ls(1) or grep(1)) is then
rebuilt with the new system files.
If you are at the last stage, and you know it (because you have looked through the
output that you were storing) then you can (fairly safely) do:
... fix the problem ...#cd /usr/src#make -DNO_CLEAN all
This will not undo the work of the previous make
buildworld.
If you see the message:
--------------------------------------------------------------
Building everything..
--------------------------------------------------------------
in the make buildworld output then it is probably fairly safe
to do so.
If you do not see that message, or you are not sure, then it is always better to be
safe than sorry, and restart the build from scratch.
23.4.14.5. How can I speed up making
the world?
Run in single user mode.
Put the /usr/src and /usr/obj
directories on separate file systems held on separate disks. If possible, put these disks
on separate disk controllers.
Better still, put these file systems across multiple disks using the ccd(4) (concatenated
disk driver) device.
Turn off profiling (set “NO_PROFILE=true” in /etc/make.conf). You almost certainly do not need it.
Also in /etc/make.conf, set CFLAGS
to something like -O -pipe. The optimization -O2 is much slower, and the optimization difference between -O and -O2 is normally negligible.
-pipe lets the compiler use pipes rather than temporary files
for communication, which saves disk access (at the expense of memory).
Pass the -jn option to
make(1) to run
multiple processes in parallel. This usually helps regardless of whether you have a
single or a multi processor machine.
The file system holding /usr/src can be mounted (or
remounted) with the noatime option. This prevents the file
system from recording the file access time. You probably do not need this information
anyway.
#mount -u -o noatime /usr/src
Warning: The example assumes /usr/src is on its own
file system. If it is not (if it is a part of /usr for example)
then you will need to use that file system mount point, and not /usr/src.
The file system holding /usr/obj can be mounted (or
remounted) with the async option. This causes disk writes to
happen asynchronously. In other words, the write completes immediately, and the data is
written to the disk a few seconds later. This allows writes to be clustered together, and
can be a dramatic performance boost.
Warning: Keep in mind that this option makes your file system more fragile.
With this option there is an increased chance that, should power fail, the file system
will be in an unrecoverable state when the machine restarts.
If /usr/obj is the only thing on this file system then it is
not a problem. If you have other, valuable data on the same file system then ensure your
backups are fresh before you enable this option.
#mount -u -o async /usr/obj
Warning: As above, if /usr/obj is not on its own file
system, replace it in the example with the name of the appropriate mount point.
23.4.14.6. What do I do if something
goes wrong?
Make absolutely sure your environment has no extraneous cruft from earlier
builds. This is simple enough.
Then restart the whole process, starting with make
buildworld.
If you still have problems, send the error and the output of uname
-a to FreeBSD general questions mailing list. Be prepared to answer other
questions about your setup!
