CVSup is a software package for distributing and updating
source trees from a master CVS repository on a remote server host. The FreeBSD sources
are maintained in a CVS repository on a central development machine in California. With
CVSup, FreeBSD users can easily keep their own source trees up
to date.
CVSup uses the so-called pull model of updating. Under the pull model, each client
asks the server for updates, if and when they are wanted. The server waits passively for
update requests from its clients. Thus all updates are instigated by the client. The
server never sends unsolicited updates. Users must either run the CVSup client manually to get an update, or they must set up a cron job to run it automatically on a regular basis.
The term CVSup, capitalized just so, refers to the entire
software package. Its main components are the client cvsup which
runs on each user's machine, and the server cvsupd which runs at
each of the FreeBSD mirror sites.
As you read the FreeBSD documentation and mailing lists, you may see references to sup. Sup was the predecessor of CVSup, and it served a similar purpose. CVSup is used much in the same way as sup and, in fact, uses
configuration files which are backward-compatible with sup's. Sup is no longer used in the FreeBSD project, because CVSup is both faster and more flexible.
Note: The csup utility is a rewrite of the CVSup software in C. Its biggest advantage is, that it is faster
and does not depend on the Modula-3 language, thus you do not need to install it as a
requirement. Moreover, if you are using FreeBSD 6.2 or later, you can use it
out-of-the-box, since it is included in the base system. Older FreeBSD versions do not
have csup(1) in their base
system but you can easily install the net/csup port, or a precompiled package. The csup utility does not support CVS mode, though. If you want to
mirror complete repositories, you will still need to use CVSup. If you decided to use csup,
just skip the steps on the installation of CVSup and
substitute the references of CVSup with csup while following the remainder of this article.
The easiest way to install CVSup is to use the precompiled
net/cvsup package from the FreeBSD packages collection. If you prefer to build CVSup from source, you can use the net/cvsup port instead. But be forewarned: the net/cvsup port depends on the Modula-3 system, which takes a
substantial amount of time and disk space to download and build.
Note: If you are going to be using CVSup on a
machine which will not have XFree86™ or Xorg installed,
such as a server, be sure to use the port which does not include the CVSup GUI, net/cvsup-without-gui.
If you want to install csup on FreeBSD 6.1 or earlier, you
can use the precompiled net/csup package from the FreeBSD packages
collection. If you prefer to build csup from source, you
can use the net/csup port instead.
CVSup's operation is controlled by a configuration file
called the supfile. There are some sample supfiles in the directory /usr/share/examples/cvsup/.
The information in a supfile answers the following questions
for CVSup:
In the following sections, we will construct a typical supfile by answering each of these questions in turn. First, we
describe the overall structure of a supfile.
A supfile is a text file. Comments begin with # and extend to the end of the line. Lines that are blank and lines
that contain only comments are ignored.
Each remaining line describes a set of files that the user wishes to receive. The line
begins with the name of a “collection”, a logical grouping of files defined
by the server. The name of the collection tells the server which files you want. After
the collection name come zero or more fields, separated by white space. These fields
answer the questions listed above. There are two types of fields: flag fields and value
fields. A flag field consists of a keyword standing alone, e.g., delete or compress. A value field also
begins with a keyword, but the keyword is followed without intervening white space by = and a second word. For example, release=cvs is a value field.
A supfile typically specifies more than one collection to
receive. One way to structure a supfile is to specify all of
the relevant fields explicitly for each collection. However, that tends to make the supfile lines quite long, and it is inconvenient because most
fields are the same for all of the collections in a supfile. CVSup provides a defaulting mechanism to avoid these problems.
Lines beginning with the special pseudo-collection name *default
can be used to set flags and values which will be used as defaults for the subsequent
collections in the supfile. A default value can be overridden
for an individual collection, by specifying a different value with the collection itself.
Defaults can also be changed or augmented in mid-supfile by additional *default lines.
With this background, we will now proceed to construct a supfile for receiving and updating the main source tree of FreeBSD-CURRENT.
Which files do you want to
receive?
The files available via CVSup are organized into named
groups called “collections”. The collections that are available are described
in the following section. In this example, we wish
to receive the entire main source tree for the FreeBSD system. There is a single large
collection src-all which will give us all of that. As a first
step toward constructing our supfile, we simply list the
collections, one per line (in this case, only one line):
src-all
Which version(s) of them do you
want?
With CVSup, you can receive virtually any version of the
sources that ever existed. That is possible because the cvsupd
server works directly from the CVS repository, which contains all of the versions. You
specify which one of them you want using the tag= and date= value fields.
Warning: Be very careful to specify any tag= fields
correctly. Some tags are valid only for certain collections of files. If you specify an
incorrect or misspelled tag, CVSup will delete files which you
probably do not want deleted. In particular, use onlytag=. for the ports-* collections.
The tag= field names a symbolic tag in the repository. There
are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific
revision. Its meaning stays the same from day to day. A branch tag, on the other hand,
refers to the latest revision on a given line of development, at any given time. Because
a branch tag does not refer to a specific revision, it may mean something different
tomorrow than it means today.
Section A.7 contains branch tags that users might be
interested in. When specifying a tag in CVSup's configuration
file, it must be preceded with tag= (RELENG_4 will become tag=RELENG_4). Keep in
mind that only the tag=. is relevant for the Ports
Collection.
Warning: Be very careful to type the tag name exactly as shown. CVSup cannot distinguish between valid and invalid tags. If you
misspell the tag, CVSup will behave as though you had
specified a valid tag which happens to refer to no files at all. It will delete your
existing sources in that case.
When you specify a branch tag, you normally receive the latest versions of the files
on that line of development. If you wish to receive some past version, you can do so by
specifying a date with the date= value field. The cvsup(1) manual
page explains how to do that.
For our example, we wish to receive FreeBSD-CURRENT. We add this line at the beginning
of our supfile:
*default tag=.
There is an important special case that comes into play if you specify neither a tag= field nor a date= field. In that case,
you receive the actual RCS files directly from the server's CVS repository, rather than
receiving a particular version. Developers generally prefer this mode of operation. By
maintaining a copy of the repository itself on their systems, they gain the ability to
browse the revision histories and examine past versions of files. This gain is achieved
at a large cost in terms of disk space, however.
Where do you want to get them
from?
We use the host= field to tell cvsup
where to obtain its updates. Any of the CVSup mirror
sites will do, though you should try to select one that is close to you in
cyberspace. In this example we will use a fictional FreeBSD distribution site, cvsup99.FreeBSD.org:
*default host=cvsup99.FreeBSD.org
You will need to change the host to one that actually exists before running CVSup. On any particular run of cvsup,
you can override the host setting on the command line, with -h hostname.
Where do you want to put them
on your own machine?
The prefix= field tells cvsup where
to put the files it receives. In this example, we will put the source files directly into
our main source tree, /usr/src. The src directory is already implicit in the collections we have chosen
to receive, so this is the correct specification:
*default prefix=/usr
Where should cvsup maintain its status files?
The CVSup client maintains certain status files in what is
called the “base” directory. These files help CVSup to work more efficiently, by keeping track of which updates
you have already received. We will use the standard base directory, /var/db:
*default base=/var/db
If your base directory does not already exist, now would be a good time to create it.
The cvsup client will refuse to run if the base directory does
not exist.
Miscellaneous supfile settings:
There is one more line of boiler plate that normally needs to be present in the supfile:
release=cvs indicates that the server should get its
information out of the main FreeBSD CVS repository. This is virtually always the case,
but there are other possibilities which are beyond the scope of this discussion.
delete gives CVSup permission to
delete files. You should always specify this, so that CVSup
can keep your source tree fully up-to-date. CVSup is careful
to delete only those files for which it is responsible. Any extra files you happen to
have will be left strictly alone.
use-rel-suffix is ... arcane. If you really want to know
about it, see the cvsup(1) manual
page. Otherwise, just specify it and do not worry about it.
compress enables the use of gzip-style compression on the
communication channel. If your network link is T1 speed or faster, you probably should
not use compression. Otherwise, it helps substantially.
As mentioned above, CVSup uses a pull method. Basically, this means that you connect to the CVSup server, and it says, “Here is what you can download
from me...”, and your client responds “OK, I will take this, this, this, and
this.” In the default configuration, the CVSup client
will take every file associated with the collection and tag you chose in the
configuration file. However, this is not always what you want, especially if you are
synching the doc, ports, or www trees -- most people cannot read four or five languages, and
therefore they do not need to download the language-specific files. If you are CVSuping the Ports Collection, you can get around this by
specifying each collection individually (e.g., ports-astrology, ports-biology, etc instead of simply saying ports-all). However, since the doc and www trees do not have
language-specific collections, you must use one of CVSup's
many nifty features: the refuse file.
The refuse file essentially tells CVSup that it should not take every single file from a
collection; in other words, it tells the client to refuse certain files from the server. The refuse file can be found (or, if you do not yet have one, should be
placed) in base/sup/. base is defined in your supfile; our defined base is
/var/db, which means that by default the refuse file is /var/db/sup/refuse.
The refuse file has a very simple format; it simply contains
the names of files or directories that you do not wish to download. For example, if you
cannot speak any languages other than English and some German, and you do not feel the
need to read the German translation of documentation, you can put the following in your
refuse file:
and so forth for the other languages (you can find the full list by browsing the FreeBSD CVS
repository).
With this very useful feature, those users who are on slow links or pay by the minute
for their Internet connection will be able to save valuable time as they will no longer
need to download files that they will never use. For more information on refuse files and other neat features of CVSup, please view its manual page.
You are now ready to try an update. The command line for doing this is quite
simple:
#cvsup supfile
where supfile is of
course the name of the supfile you have just created. Assuming
you are running under X11, cvsup will display a GUI window with
some buttons to do the usual things. Press the go button,
and watch it run.
Since you are updating your actual /usr/src tree in this
example, you will need to run the program as root so that cvsup has the permissions it needs to update your files. Having just
created your configuration file, and having never used this program before, that might
understandably make you nervous. There is an easy way to do a trial run without touching
your precious files. Just create an empty directory somewhere convenient, and name it as
an extra argument on the command line:
#mkdir /var/tmp/dest#cvsup supfile /var/tmp/dest
The directory you specify will be used as the destination directory for all file
updates. CVSup will examine your usual files in /usr/src, but it will not modify or delete any of them. Any file
updates will instead land in /var/tmp/dest/usr/src. CVSup will also leave its base directory status files untouched
when run this way. The new versions of those files will be written into the specified
directory. As long as you have read access to /usr/src, you do
not even need to be root to perform this kind of trial run.
If you are not running X11 or if you just do not like GUIs, you should add a couple of
options to the command line when you run cvsup:
#cvsup -g -L 2 supfile
The -g tells CVSup not to use
its GUI. This is automatic if you are not running X11, but otherwise you have to specify
it.
The -L 2 tells CVSup to print
out the details of all the file updates it is doing. There are three levels of verbosity,
from -L 0 to -L 2. The default is
0, which means total silence except for error messages.
There are plenty of other options available. For a brief list of them, type cvsup -H. For more detailed descriptions, see the manual page.
Once you are satisfied with the way updates are working, you can arrange for regular
runs of CVSup using cron(8). Obviously,
you should not let CVSup use its GUI when running it from cron(8).
The file collections available via CVSup are organized
hierarchically. There are a few large collections, and they are divided into smaller
sub-collections. Receiving a large collection is equivalent to receiving each of its
sub-collections. The hierarchical relationships among collections are reflected by the
use of indentation in the list below.
The most commonly used collections are src-all, and ports-all. The other collections are used only by small groups of
people for specialized purposes, and some mirror sites may not carry all of them.
cvs-all release=cvs
The main FreeBSD CVS repository, including the cryptography code.
distrib release=cvs
Files related to the distribution and mirroring of FreeBSD.
doc-all release=cvs
Sources for the FreeBSD Handbook and other documentation. This does not include files
for the FreeBSD web site.
ports-all release=cvs
The FreeBSD Ports Collection.
Important: If you do not want to update the whole of ports-all (the whole ports tree), but use one of the subcollections
listed below, make sure that you always update the ports-base
subcollection! Whenever something changes in the ports build infrastructure represented
by ports-base, it is virtually certain that those changes will
be used by “real” ports real soon. Thus, if you only update the
“real” ports and they use some of the new features, there is a very high
chance that their build will fail with some mysterious error message. The very first thing to do in this case is to
make sure that your ports-base subcollection is up to date.
Important: If you are going to be building your own local copy of ports/INDEX, you must accept ports-all (the whole
ports tree). Building ports/INDEX with a partial tree is not
supported. See the FAQ.
ports-accessibility release=cvs
Software to help disabled users.
ports-arabic release=cvs
Arabic language support.
ports-archivers release=cvs
Archiving tools.
ports-astro release=cvs
Astronomical ports.
ports-audio release=cvs
Sound support.
ports-base release=cvs
The Ports Collection build infrastructure - various files located in the Mk/ and Tools/ subdirectories of /usr/ports.
Note: Please see the important
warning above: you should always update this subcollection, whenever you update any
part of the FreeBSD Ports Collection!
ports-benchmarks release=cvs
Benchmarks.
ports-biology release=cvs
Biology.
ports-cad release=cvs
Computer aided design tools.
ports-chinese release=cvs
Chinese language support.
ports-comms release=cvs
Communication software.
ports-converters release=cvs
character code converters.
ports-databases release=cvs
Databases.
ports-deskutils release=cvs
Things that used to be on the desktop before computers were invented.
ports-devel release=cvs
Development utilities.
ports-dns release=cvs
DNS related software.
ports-editors release=cvs
Editors.
ports-emulators release=cvs
Emulators for other operating systems.
ports-finance release=cvs
Monetary, financial and related applications.
ports-ftp release=cvs
FTP client and server utilities.
ports-games release=cvs
Games.
ports-german release=cvs
German language support.
ports-graphics release=cvs
Graphics utilities.
ports-hebrew release=cvs
Hebrew language support.
ports-hungarian release=cvs
Hungarian language support.
ports-irc release=cvs
Internet Relay Chat utilities.
ports-japanese release=cvs
Japanese language support.
ports-java release=cvs
Java™ utilities.
ports-korean release=cvs
Korean language support.
ports-lang release=cvs
Programming languages.
ports-mail release=cvs
Mail software.
ports-math release=cvs
Numerical computation software.
ports-mbone release=cvs
MBone applications.
ports-misc release=cvs
Miscellaneous utilities.
ports-multimedia release=cvs
Multimedia software.
ports-net release=cvs
Networking software.
ports-net-im release=cvs
Instant messaging software.
ports-net-mgmt release=cvs
Network management software.
ports-net-p2p release=cvs
Peer to peer networking.
ports-news release=cvs
USENET news software.
ports-palm release=cvs
Software support for Palm™ series.
ports-polish release=cvs
Polish language support.
ports-ports-mgmt release=cvs
Utilities to manage ports and packages.
ports-portuguese release=cvs
Portuguese language support.
ports-print release=cvs
Printing software.
ports-russian release=cvs
Russian language support.
ports-science release=cvs
Science.
ports-security release=cvs
Security utilities.
ports-shells release=cvs
Command line shells.
ports-sysutils release=cvs
System utilities.
ports-textproc release=cvs
text processing utilities (does not include desktop publishing).
ports-ukrainian release=cvs
Ukrainian language support.
ports-vietnamese release=cvs
Vietnamese language support.
ports-www release=cvs
Software related to the World Wide Web.
ports-x11 release=cvs
Ports to support the X window system.
ports-x11-clocks release=cvs
X11 clocks.
ports-x11-drivers release=cvs
X11 drivers.
ports-x11-fm release=cvs
X11 file managers.
ports-x11-fonts release=cvs
X11 fonts and font utilities.
ports-x11-toolkits release=cvs
X11 toolkits.
ports-x11-servers release=cvs
X11 servers.
ports-x11-themes release=cvs
X11 themes.
ports-x11-wm release=cvs
X11 window managers.
projects-all release=cvs
Sources for the FreeBSD projects repository.
src-all release=cvs
The main FreeBSD sources, including the cryptography code.
src-base release=cvs
Miscellaneous files at the top of /usr/src.
src-bin release=cvs
User utilities that may be needed in single-user mode (/usr/src/bin).
src-cddl release=cvs
Utilities and libraries covered by the CDDL license (/usr/src/cddl).
src-contrib release=cvs
Utilities and libraries from outside the FreeBSD project, used relatively unmodified
(/usr/src/contrib).
src-crypto release=cvs
Cryptography utilities and libraries from outside the FreeBSD project, used relatively
unmodified (/usr/src/crypto).
src-eBones release=cvs
Kerberos and DES (/usr/src/eBones). Not used in current
releases of FreeBSD.
src-etc release=cvs
System configuration files (/usr/src/etc).
src-games release=cvs
Games (/usr/src/games).
src-gnu release=cvs
Utilities covered by the GNU Public License (/usr/src/gnu).