Task: Define Test Strategy
This task describes how to define the test strategy and the specific techniques that will be employed, and outlines the test automation architecture.
Disciplines: Test
Purpose

The purpose of this task is to:

  • Identify each specific technique that will be employed to enable the desired testing
  • Outline the workings of each technique, including the types of testing that it supports
  • Define a candidate architecture for the test automation system
Relationships
Steps
Examine Test Motivators and Test Items

Using the evaluation mission as context, examine the test-related plans, and study the test motivators that have been identified for the forthcoming test effort. It may be necessary to do further investigation at the motivator source: usually the iteration plan provides a means of locating additional information.

For each motivator, consider what test approach and associated techniques might be required to address each motivator. Also examine the Test Plan and study the test items. Each targeted test item should be considered in relation to each Motivator, and the approach and techniques extended accordingly. If you cannot find a lot of detail about (or you are unfamiliar with) the test items, it may be useful to discuss the targeted items with the development staff, usually by starting with the software architect or development team leads.

Focus on identifying the minimal set of techniques necessary to satisfactorily address the evaluation mission and motivators. Look for opportunities where one technique can be used to address more than one aspect of the required testing. Note other potential techniques that seem interesting to explore, but be able to identify these as additional rather than essential.

Examine the Software Architecture

Study the Software Architecture to gain an understanding of its key elements: mechanisms, main views, and so on. Typically, the Software Architecture documentation provides good information focused at the right level of detail for use in considering a test approach. To clarify its information, or in the absence of a document, it is useful to discuss the architecture with the development staff, usually by talking to the software architect directly, or one of the development team leads.

Focus on identifying and discussing the key mechanisms, and gaining a good understanding of these aspects of the system. Each mechanism and key feature of the architecture will likely present challenges or constraints for the test effort. For example, a distributed architecture may necessitate organizing the test team into sub-teams, each team targeting an architectural tier.

While a creative way to the test implementation and execution strategy can often be used to overcome these challenges, it may be necessary to have the development team modify the software to enable testing, as discussed in Task: Define Testability Elements.

Consider the Appropriate Breadth and Depth of the Test Approach
Considering all of the details that are now known about the requirements on the test approach, it is beneficial to step back and consider the test approach from a higher-level perspective. What things does the test approach not address that it should? Are there any concerns that should be explored that do not appear in any of the documented information?

Based on your experience, review the requirements for the test approach for appropriate breadth and depth for this stage in the project lifecycle. Consider additional requirements that will help to present a more complete approach.

Identify Existing Test Techniques for Reuse
From your own experience, or other experience that you have access to, identify existing techniques that will either meet the requirements of the test approach, or can be adapted to meet them.
Identify Additional Techniques

It is not terribly useful to think in terms of a complete test approach: there are always additional techniques that you might try if you only had limitless time and resources.

However, it is important that the test approach is well-rounded and comprehensive enough to allow a useful evaluation of perceived quality to be made. This requires an approach that evaluates sufficient aspects of quality risk (or dimensions of quality) for the project team to assess perceived quality with a justified degree of confidence.

Define Techniques

Outline the workings of each technique. Address the type of testing that it supports, and the objective, scope, implementation method, test oracles, assessment method, and automation needs of the technique.

In many cases, you will reuse technique from one project to the next. In this situation, you can simply reference a common definition of the technique, or copy the existing definition and revise it as appropriate.

For more details, see: Guideline: Defining Test Techniques.

Outline the Test Automation Architecture

Based on experience gained from similar systems or in similar problem domains, begin to define a candidate architecture for the test automation system.

Define the Test Asset Configuration Management Strategy

Like many other work products produced during a software development project, test assets are candidates for configuration management and version control.

The specific requirements can range in complexity from the decision to use basic backup and recovery services enabled, to having full support for parallel development of automated Test Scripts at multiple sites against different versions of an application.

Give thought to your requirements for configuration management, and begin to outline probable logistical needs to realize those requirements.

Survey Availability of Reusable Assets

Sometimes, it makes sense to build assets from scratch, and sometimes it does not. Try to find a good balance between a complete "roll-you-own" philosophy and establishing a rigid and bureaucratic librarian policy on new work product creation.

There are times when one approach is better than the other, and you should be flexible enough to take advantage of the benefits that both approaches bring.

Capture Your Findings

Depending on a number of factors (including team size and organization culture), there will be better and worse ways to record the decisions that you have made about the test approach.

You will typically have two audiences to consider: the management team will want to review this information to provide approval and be aware of the logistics implications of the approach, and the test team will want to use the test approach as guidance for the work that they undertake. Try to find an appropriate medium to suitably address both needs (perhaps using a project Intranet Web site).

Evaluate and Verify Your Results

You should evaluate whether your work is of appropriate quality, and that it is complete enough to be useful to those team members who will make subsequent use of it as input to their work. Where possible, use checklists to verify that quality and completeness are good enough.

Have the people who perform the downstream tasks that rely on your work as input review your interim work. Do this while you still have time available to address their concerns. You should also evaluate your work against the key input work products to make sure that you have represented them accurately and sufficiently. It may be useful to have the author of the input work product review your work on this basis.

More Information