Testing is applied to different types of targets, in different stages or levels of work effort. These levels are
distinguished typically by those roles that are best skilled to design and conduct the tests, and where techniques are
most appropriate for testing at each level. It's important to ensure a balance of focus is retained across these
different work efforts.
Developer Testing
Developer testing denotes the aspects of test design and implementation most appropriate for the team of developers to
undertake; this is in contrast to independent testing.
Traditionally, developer testing has been considered mainly with respect to unit testing. While some developers also
perform varying levels integration testing, this is largely dependent on culture and other context issues.
Independent Testing
Independent testing denotes the test design and implementation most appropriately performed by someone who is
independent from the team of developers. You can consider this distinction a superset, which includes Independent
Verification & Validation. In most cases, test execution initially occurs with the independent testing group that
designed and implemented the test, but the independent testers should create their tests to make them available to the
developer testing groups for execution. Boris Beizer gives the following explanation of the different objective that
independent testing has over developer testing:
"The purpose of independent testing is to provide a different perspective and, therefore, different tests;
furthermore to conduct those tests in a richer [...] environment than is possible for the developer." [Boris Beizer
1995. Black Box Testing. New York, NY: John Wiley & Sons, Inc]
Independent Stakeholder Testing
An alternate view of independent testing is that it represents testing that is based on the needs and concerns of
various stakeholders. Therefore it's referred to as Stakeholder Testing. This is an important distinction-it helps to
include a broader set of stakeholder concerns than might traditionally be considered, extending the somewhat generic
"customer" with stakeholders such as technical support staff, technical trainers, sales staff in additional to
customers, and users.
As a final comment, Extreme Programming's notion of customer tests relates to this categorization of
independent testing.
Unit Testing
Unit testing focuses on verifying the smallest testable elements of the software. Typically unit testing is applied to
components represented in the implementation model to verify that control flows and data flows are covered, and that
they function as expected. The developers perform unit testing as the unit is developed.
Integration Testing
Integration testing is performed to ensure that the components in the implementation model operate properly when
combined. The target-of-test is a package or a set of packages in the implementation model. Often the packages being
combined come from different development organizations. Integration testing exposes incompleteness or mistakes in the
package's interface specifications.
In some cases, the assumption by developers is that other groups such as independent testers will perform integration
tests. This situation presents risks to the software project and ultimately the software quality because:
-
integration areas are a common point of software failure.
-
integration tests performed by independent testers typically use black-box techniques and are typically dealing
with larger software components.
A better approach is to consider integration testing the responsibility of both developer and independent testers, but
make sure that the strategies of each team testing efforts do not overlap significantly. The exact nature of that
overlap is based on the needs of the individual project. We recommend you foster an environment where developers and
independent system testers share a single vision of quality. See Concept: Developer Testing for additional information.
System Testing
Traditionally system testing is done when the software is functioning as a whole. An iterative lifecycle allows system
testing to occur much earlier-as soon as well-formed subsets of the use-case behavior are implemented. Usually the
target is the system's end-to-end functioning elements.
Acceptance Testing
User acceptance testing is the final test action taken before deploying the software. The goal of acceptance
testing is to verify that the software is ready, and that it can be used by users to perform those functions and tasks
for which the software was built. See Concept: Acceptance Testing for additional information.
There are other notions of acceptance testing, which are generally characterized by a hand-off from one group or one
team to another. For example, a build acceptance test is the testing done to accept the hand-over of a new
software build from development into independent testing.
A comment about sequence and timing of test levels
Traditionally, unit testing is thought of as being implemented early, as the first stage of testing: all units required
to be passed before subsequent stages are conducted. However, in an iterative development process, this approach is as
a general rule, inappropriate. A better approach is to identify the unit, integration and system tests that offer most
potential for finding errors, then implement and execute them based on a combination of greatest risk and supporting
environment.
|