Disk space is one of the most important requirements. Depending on the set of releases, architectures, and degree of completeness you want to mirror, a huge amount of disk space may be consumed. Also keep in mind that official mirrors are probably required to be complete. The CVS repository and the web pages should always be mirrored completely. Also note that the numbers stated here are reflecting the current state (at 8.4-RELEASE/9.1-RELEASE). Further development and releases will only increase the required amount. Also make sure to keep some (ca. 10-20%) extra space around just to be sure. Here are some approximate figures:
Full FTP Distribution: 1.0 TB
CVS repository: 5.4 GB
CTM deltas: 3.2 GB
Web pages: 463 MB
The current disk usage of FTP Distribution can be found at ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes.
Of course, you need to be connected to the Internet. The required bandwidth depends on your intended use of the mirror. If you just want to mirror some parts of FreeBSD for local use at your site/intranet, the demand may be much smaller than if you want to make the files publicly available. If you intend to become an official mirror, the bandwidth required will be even higher. We can only give rough estimates here:
Local site, no public access: basically no minimum, but < 2 Mbps could make syncing too slow.
Unofficial public site: 34 Mbps is probably a good start.
Official site: > 100 Mbps is recommended, and your host should be connected as close as possible to your border router.
One thing this depends on the expected number of clients, which is determined by the server's policy. It is also affected by the types of services you want to offer. Plain FTP or HTTP services may not require a huge amount of resources. Watch out if you provide rsync. This can have a huge impact on CPU and memory requirements as it is considered a memory hog. The following are just examples to give you a very rough hint.
For a moderately visited site that offers Rsync, you might consider a current CPU with around 800MHz - 1 GHz, and at least 512MB RAM. This is probably the minimum you want for an official site.
For a frequently used site you definitely need more RAM (consider 2GB as a good start) and possibly more CPU, which could also mean that you need to go for a SMP system.
You also want to consider a fast disk subsystem. Operations on the CVS repository require a fast disk subsystem (RAID is highly advised). A SCSI controller that has a cache of its own can also speed up things since most of these services incur a large number of small modifications to the disk.
Every mirror site is required to have a set of core services available. In addition to these required services, there are a number of optional services that server administrators may choose to offer. This section explains which services you can provide and how to go about implementing them.
This is one of the most basic services, and
it is required for each mirror offering public
FTP distributions. FTP access must be
anonymous, and no upload/download ratios
are allowed (a ridiculous thing anyway).
Upload capability is not required (and must
never be allowed for the FreeBSD file space).
Also the FreeBSD archive should be available under
the path /pub/FreeBSD
.
There is a lot of software available which can be set up to allow anonymous FTP (in alphabetical order).
/usr/libexec/ftpd
: FreeBSD's own ftpd
can be used. Be sure to read ftpd(8).
ftp/ncftpd
: A commercial package,
free for educational use.
ftp/oftpd
: An ftpd designed with
security as a main focus.
ftp/proftpd
: A modular and very flexible ftpd.
ftp/pure-ftpd
: Another ftpd developed with
security in mind.
ftp/twoftpd
: As above.
ftp/vsftpd
: The “very secure” ftpd.
ftp/wu-ftpd
: The ftpd from Washington
University. It has become infamous, because of the huge
amount of security issues that have been found in it.
If you do choose to use this software be sure to
keep it up to date.
FreeBSD's ftpd, proftpd, wu-ftpd and maybe ncftpd are among the most commonly used FTPds. The others do not have a large userbase among mirror sites. One thing to consider is that you may need flexibility in limiting how many simultaneous connections are allowed, thus limiting how much network bandwidth and system resources are consumed.
Rsync is often offered for access to the
contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The
protocol is different from FTP in many ways.
It is much more
bandwidth friendly, as only differences between files
are transferred instead of whole files when they change.
Rsync does require a significant amount of memory for
each instance. The size depends on the size of
the synced module in terms of the number of directories and
files. Rsync can use rsh
and
ssh
(now default) as a transport,
or use its own protocol for stand-alone access
(this is the preferred method for public rsync servers).
Authentication, connection limits, and other restrictions
may be applied. There is just one software package
available:
net/rsync
If you want to offer the FreeBSD web pages, you will need to install a web server. You may optionally offer the FTP fileset via HTTP. The choice of web server software is left up to the mirror administrator. Some of the most popular choices are:
www/apache22
:
Apache is the most widely
deployed web server on the Internet. It is used
extensively by the FreeBSD Project.
www/thttpd
:
If you are going to be serving a large amount of static content
you may find that using an application such as thttpd is more
efficient than Apache. It is
optimized for excellent performance on FreeBSD.
www/boa
:
Boa is another alternative to
thttpd and
Apache. It should provide
considerably better performance than
Apache for purely static
content. It does not, at the time of this writing,
contain the same set of optimizations for FreeBSD that
are found in thttpd.
CVSup is a very efficient way of distributing files.
It works similar to rsync, but was specially designed for
use with CVS repositories. If you want to offer the
FreeBSD CVS repository, you really want to consider
offering it via CVSup. It is possible to offer
the CVS repository via AnonCVS, FTP,
rsync or HTTP, but
people would benefit much more from CVSup access.
CVSup was developed by John Polstra <jdp@FreeBSD.org>
.
It is a bit tricky to install on non-FreeBSD platforms,
since it is written in Modula-3 and therefore requires
a Modula-3 environment. John Polstra has built a
stripped down version of M3 that is sufficient to
run CVSup, and can be installed much easier.
See Ezm3
for details. Related ports are:
net/cvsup
: The native CVSup port (client and server)
which requires lang/ezm3
now.
net/cvsup-mirror
: The CVSup mirror kit, which requires
net/cvsup-without-gui
, and configures it mirror-ready. Some
site administrators may want a different setup though.
There are a few more like
net/cvsup-without-gui
you might want to have
a look at. If you prefer a static binary package, take a look
here.
This page still refers to the S1G bug that was present
in CVSup. Maybe
John will set up a generic download-site to get
static binaries for various platforms.
It is possible to use CVSup to offer any kind of fileset, not just CVS repositories, but configuration can be complex. CVSup is known to eat some CPU on both the server and the client, since it needs to compare lots of files.
If you have the CVS repository, you may want to offer anonymous CVS access. A short warning first: There is not much demand for it, it requires some experience, and you need to know what you are doing.
Generally there are two ways
to access a CVS repository remotely: via
pserver or via ssh
(we do not consider rsh
).
For anonymous access, pserver is
very well suited, but some still offer ssh
access as well. There is a custom crafted
wrapper
in the CVS repository, to be used as a login-shell for the
anonymous ssh account. It does a chroot, and therefore
requires the CVS repository to be available under the
anonymous user's home-directory. This may not be possible
for all sites. If you just offer pserver
this restriction does not apply, but you may run with
more security risks. You do not need to install any special
software, since cvs(1) comes with
FreeBSD. You need to enable access via inetd
,
so add an entry into your /etc/inetd.conf
like this:
See the manpage for details of the options. Also see the CVS info
page about additional ways to make sure access is read-only.
It is advised that you create an unprivileged account,
preferably called anoncvs
.
Also you need to create a file passwd
in your /home/ncvs/CVSROOT
and assign a
CVS password (empty or anoncvs
) to that user.
The directory /anoncvstmp
is a special
purpose memory based file system. It is not required but
advised since cvs(1) creates a shadow directory
structure in your /tmp
which is
not used after the operation but slows things
dramatically if real disk operations are required.
Here is an excerpt from /etc/fstab
,
how to set up such a MFS:
This is (of course) tuned a lot, and was suggested by John Polstra <jdp@FreeBSD.org>
.
This, and other documents, can be downloaded from http://ftp.FreeBSD.org/pub/FreeBSD/doc/
For questions about FreeBSD, read the
documentation before
contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.