Please talk to Mark Linimon before making any changes to this section.
As of January 2013, a rewrite is in progress to further separate privileges. The following concepts are introduced:
Server-side user portbuild assumes all responsiblity for operations involving builds and communicating with the clients. This user no longer has access to sudo.
Server-side user srcbuild is created and given responsiblity for operations involving both VCS operations and anything involving src builds for the clients. This user does not have access to sudo.
The server-side ports-arch users go away.
None of the above server-side users have ssh keys. Individual portmgr will accomplish all those tasks using ksu. (This is still work-in-progress.)
The only client-side user is also named portbuild and still has access to sudo for the purpose of managing jails.
This document has not yet been updated with the latest changes.
Install FreeBSD.
Create a user to own the portbuild repository, such as portbuild. It should have the '*' password.
Export that value for a later initialization step:
# export PORTBUILD_USER=portbuild
Similarly, create a user to own the svn repository, such as srcbuild. It should have the '*' password.
Export that value for a later initialization step:
# export SRCBUILD_USER=srcbuild
Add the following to /boot/loader.conf:
console="vidconsole,comconsole"
You should run the cluster on UTC. If you have not set the clock to UTC:
# cp -p /usr/share/zoneinfo/Etc/UTC /etc/localtime
Create the appropriate /etc/rc.conf.
Required entries:
hostname="${hostname}"
sshd_enable="YES"
zfs_enable="YES"
Recommended entries:
background_fsck="NO" clear_tmp_enable="YES" dumpdev="AUTO" fsck_y_enable="YES" apache22_enable="YES" apache_flags="" apache_pidfile="/var/run/httpd.pid" gmetad_enable="YES" gmond_enable="YES" inetd_enable="YES" inetd_flags="-l -w" mountd_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 12" nfs_remote_port_only="YES" ntpd_enable="YES" rpcbind_enable="YES" rpc_lockd_enable="NO" rpc_statd_enable="YES" sendmail_enable="NONE" smartd_enable="YES"
Create /etc/resolv.conf, if necessary.
Create the appropriate files in /etc/ssh/.
Add the following to /etc/sysctl.conf:
kern.maxfiles=40000 kern.maxfilesperproc=38000 sysctl vfs.usermount=1 sysctl vfs.zfs.super_owner=1
Make sure the following change is made to /etc/ttys:
ttyu0 "/usr/libexec/getty std.9600" vt100 on secure
You should be able to install from the most recent release using only the default kernel configuration.
The following ports (or their latest successors) are required:
databases/py-sqlite3 databases/py-sqlalchemy (only SQLITE is needed) devel/git (WITH_SVN) devel/py-configobj devel/py-setuptools devel/subversion net/nc net/rsync sysutils/ganglia-monitor-core (with GMETAD off) sysutils/ganglia-webfrontend (compile with -DWITHOUT_X11) www/apache22 (with EXT_FILTER)
Expect those to bring in, among others:
databases/sqlite3 lang/perl-5.14 (or successor) lang/python27 (or sucessor)
The following ports (or their latest successors) are strongly suggested:
devel/ccache mail/postfix net/isc-dhcp41-server ports-mgmt/pkg ports-mgmt/portaudit ports-mgmt/portmaster shells/bash shells/zsh sysutils/screen
Note: The use of sudo on the master, which was formerly required, is no longer recommended.
The following ports (or their latest successors) are handy:
benchmarks/bonnie++ ports-mgmt/pkg_tree sysutils/dmidecode sysutils/smartmontools sysutils/zfs-stats
The following steps need to be done as euid root.
Pick a zfs volume name and export it. We have used a so far to date.
# export ZFS_VOLUME=a
Pick a mountpoint and export it. We have used /a so far to date.
# export ZFS_MOUNTPOINT=/a
Create the zfs volume and mount it.
Example 10. Creating a zfs volume for portbuild
# zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
Note: We will define a zfs permission set below, so that the portbuild user may administer this volume without having to have root privileges.
Select an svn repository and export it. See the FreeBSD Handbook for the currently supported list.
# export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org
Obtain a copy of the kickstart script into a temporary directory. (You will not need to keep this directory later.)
# mkdir -p /home/portbuild/tmp
# svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/admin/tools /home/portbuild/tmp
Run the kickstart script:
# sh /home/portbuild/tmp/mkportbuild
This will accomplish all the following 5 steps:
Create the portbuild directory:
# mkdir -p ${ZFS_MOUNTPOINT}/portbuild
Create and mount a new zfs filesystem on it:
zfs create -o mountpoint=${ZFS_MOUNTPOINT}/portbuild ${ZFS_VOLUME}/portbuild
Set up the directory:
# chown ${PORTBUILD_USER}:${PORTBUILD_USER} ${ZFS_MOUNTPOINT}/portbuild
# chmod 775 ${ZFS_MOUNTPOINT}/portbuild
# ln -sf ${ZFS_MOUNTPOINT}/portbuild /var/portbuild
Note: The ln is necessary due to a number of hardcoded paths. This is a bug.
Set up the initial repository:
% svn checkout ${VCS_REPOSITORY}/base/projects/portbuild ${ZFS_MOUNTPOINT}/portbuild
Set up the zfs permission sets.
Configure how build slaves will talk to your server by making the following changes to /a/portbuild/conf/client.conf:
Set CLIENT_NFS_MASTER to wherever your build slaves will PXE boot from. (Possibly, the hostname of your server.)
Set CLIENT_BACKUP_FTP_SITE to a backup site for FTP fetches; again, possibly the hostname of your server.
Set CLIENT_UPLOAD_HOST to where completed packages will be uploaded.
Most of the other default values should be fine.
Most of the default values in /a/portbuild/conf/common.conf should be fine. This file holds definitions used by both the server and all its clients.
Configure the server by making the following changes to /a/portbuild/conf/server.conf:
Set SUPPORTED_ARCHS to the list of architectures you wish to build packages for.
For each source branch you will be building for, set SRC_BRANCHES and SRC_BRANCH_branch_SUBDIR as detailed in Section 14.1. You should not need to change SRC_BRANCHES_PATTERN.
Set ZFS_VOLUME and ZFS_MOUNTPOINT to whatever you chose above.
Set UPLOAD_DIRECTORY, UPLOAD_TARGET, and UPLOAD_USER as appropriate for your site.
Set VCS_REPOSITORY to whatever you chose above.
Set MASTER_URL to the http URL of your server. This will be stamped into the package build logs and the indices thereof.
Most of the other default values should be fine.
For each architecture, follow the steps in Section 17.1.
Copy the following files from /a/portbuild/admin/etc/rc.d/ to /usr/local/etc/rc.d/:
pollmachine qmanager
As root, start each one of them. You may find it handy to start each under screen for debugging purposes.
Initialize the qmanager database's acl list:
Note: This should now be automatically done for you by the first build command.
# python /a/portbuild/qmanager/qclient add_acl name=deny_all uidlist= gidlist= sense=0
As the srcbuild user, run the following commands manually to create the src and ports repositories, respectively:
% /a/portbuild/admin/scripts/updatesnap.ports % /a/portbuild/admin/scripts/updatesnap
These will be periodically run from the srcbuild crontab, which you will install below.
Configure /usr/local/etc/apache22/httpd.conf as appropriate for your site.
Copy /a/portbuild/admin/conf/apache.conf to the appropriate Includes/ subdirectory, e.g., /usr/local/etc/apache22/Includes/portbuild.conf. Configure it as appropriate for your site.
Install /a/portbuild/admin/crontabs/portbuild as the portbuild crontab via crontab -u portbuild -e. If you do not support all the archs listed there, make sure to comment out the appropriate dologs entries.
Install /a/srcbuild/admin/crontabs/portbuild as the srcbuild crontab via crontab -u srcbuild -e.
If your build slaves will be pxebooted, make sure to enable the tftp entries in /etc/inetd.conf.
Configure mail by doing the following:
newaliases.
For each architecture, follow the steps in Section 17.2.
You will probably find it handy to append the following to the PATH definition for the portbuild user:
/a/portbuild/scripts:/a/portbuild/tools
You will also probably find it handy to append the following to the PATH definition for the srcbuild user:
/a/portbuild/admin/scripts:/a/portbuild/admin/tools
You should now be ready to build packages.