This document contains information about testing the release candidates that will ultimately be the next LLVM release. For more information on how to manage the actual release, please refer to How To Release LLVM To The Public.
Once the release process starts, the Release Manager will ask for volunteers, and it’ll be the role of each volunteer to:
Not all bugs or regressions are show-stoppers and it’s a bit of a grey area what should be fixed before the next candidate and what can wait until the next release.
It’ll depend on:
The scripts are in the utils/release directory.
This script will configure and compile LLVM+Clang (+ most add-ons, like compiler-rt, libcxx and clang-extra-tools) in three stages, and will test the final stage. It’ll have installed the final binaries on the Phase3/Release+Asserts directory, and that’s the one you should use for the test-suite and other external tests.
Tip: For some reason, the script bails on ARM because of path mismatch on configure. The solution was to add a symlink inside the rc1 directory to the llvm.src directory:
mkdir rc1
cd rc1
ln -s ../../llvm.src
cd ..
To run the script, you must be on the source directory and pass the right options, for example:
./utils/release/test-release.sh \
-release 3.3 \
-rc 1 \
-no-checkout \
-no-64bit \
-test-asserts \
-no-compare-files
Each system will require different options. For instance, x86_64 will obviously not need -no-64bit while 32-bit systems will, or the script will fail.
The important flags to get right are:
This script builds three phases of Clang+LLVM twice each (Release and Release+Asserts), so use screen or nohup to avoid headaches, since it’ll take a long time.
Use the --help option to see all the options and chose it according to your needs.
TODO
Follow the LNT Quick Start Guide link on how to set-up the test-suite
An example on the run command line, assuming you created a link from the correct install directory to ~/devel/llvm/install:
./sandbox/bin/python sandbox/bin/lnt runtest \
nt \
-j4 \
--sandbox sandbox \
--test-suite ~/devel/llvm/test/test-suite \
--cc ~/devel/llvm/install/bin/clang \
--cxx ~/devel/llvm/install/bin/clang++
When the release process is announced on the mailing list, you should prepare for the testing, by applying the same testing you’ll do on the release candidates, on the previous release.
You should:
If the final phase’s make check-all failed, it’s a good idea to also test the intermediate stages by going on the obj directory and running make check-all to find if there’s at least one stage that passes (helps when reducing the error for bug report purposes).
When the Release Manager sends you the release candidate, download all sources, unzip on the same directory (there will be sym-links from the appropriate places to them), and run the release test as above.
You should:
Once the release manages announces that the latest candidate is the good one, you have to pack the Release (no Asserts) install directory on Phase3 and that will be the official binary.
If you found regressions or failures when comparing a release candidate with the previous release, follow the rules below: