7. Build Maintenance
Prev |  | Next

## 7. Build Maintenance

There are several cases where you will need to manually clean up a build:

1. You have manually interrupted it.

2. The head node has been rebooted while a build was running.

3. `qmanager` has crashed and has been restarted.

### 7.1. Interrupting a Build

Manually interrupting a build is a bit messy. First you need to identify the tty in which it's running (either record the output of tty(1) when you start the build, or use `ps x` to identify it. You need to make sure that nothing else important is running in this tty, e.g., `ps -t p1` or whatever. If there is not, you can just kill off the whole term easily with `pkill -t pts/1`; otherwise issue a `kill -HUP` in there by, for example, `ps -t pts/1 -o pid= | xargs kill -HUP`. Replace `p1` by whatever the tty is, of course.

The package builds dispatched by `make` to the client machines will clean themselves up after a few minutes (check with `ps x` until they all go away).

If you do not kill make(1), then it will spawn more jobs. If you do not kill `dopackages`, then it will restart the entire build. If you do not kill the `pdispatch` processes, they'll keep going (or respawn) until they've built their package.

### 7.2. Cleaning up a Build

To free up resources, you will need to clean up client machines by running `build cleanup` command. For example:

`%` `/a/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full`

If you forget to do this, then the old build `jail`s will not be cleaned up for 24 hours, and no new jobs will be dispatched in their place since `pointyhat` thinks the job slot is still occupied.

To check, `cat ~/loads/*` to display the status of client machines; the first column is the number of jobs it thinks is running, and this should be roughly concordant with the load average. `loads` is refreshed every 2 minutes. If you do `ps x | grep pdispatch` and it is less than the number of jobs that `loads` thinks are in use, you are in trouble.

### Note:

The following notes about mounting only apply to `connected` nodes.

You may have problem with the `umount` commands hanging. If so, you are going to have to use the `allgohans` script to run an ssh(1) command across all clients for that buildenv. For example:

`%` ssh gohan24 df

will get you a df, and

`%` allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports" `%` allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/src"

are supposed to get rid of the hanging mounts. You will have to keep doing them since there can be multiple mounts.

### Note:

Ignore the following:

umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: statfs: No such file or directory umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: unknown file system umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed! /x/tmp/8-exp/chroot/53837/compat/linux/proc: not a file system root directory

The former two mean that the client did not have those mounted; the latter two are a bug.

You may also see messages about `procfs`.

### Note:

The above is the end of the notes that apply only to `connected` nodes.

After you have done all the above, remove the ``${arch}`/lock` file before trying to restart the build. If you do not, `dopackages` will simply exit.

If you have to do a ports tree update before restarting, you may have to rebuild either `duds`, `INDEX`, or both.

### 7.3. Maintaining builds with the `build` command

Here are the rest of the options for the `build` command:

- `build destroy `arch` `branch`` - Destroy the build id.

- `build list `arch` `branch`` - Shows the current set of build ids.

Prev |  | Next
6. Anatomy of a Build | Home | 8. Monitoring the Build

All FreeBSD documents are available for download at http://ftp.FreeBSD.org/pub/FreeBSD/doc/

Questions that are not answered by the documentation may be sent to <freebsd-questions@FreeBSD.org>.
Send questions about this document to <freebsd-doc@FreeBSD.org>.