Task: Perform Review
This task examines a work product to ensure that it meets the need and intent of the work product and conforms to relevant standards and guidelines, during a review meeting.
Disciplines: Environment
Purpose

The purpose of this task is to assess the quality, content, and/or compliance of a coherent set of work products with a focus on:

  • Improve the content or organization of the work product
  • Disseminate information amongst a set of stakeholders
  • Ensure adherence to QA standards and guidelines
  • Ensure adherence to relevant industry guidelines and practices
  • Ensure adherence to standards
Relationships
RolesPrimary Performer: Additional Performers:
InputsMandatory:
  • None
Optional:
    Outputs
      Main Description

      There are a number of work products created in a modern software-centric development project. Any of them may be subject to a review meeting, depending on the delivery process in which they are created. Some of these include:

      • Requirements specification
      • Systems engineering models
      • Architectural Specifications
      • Project schedules
      • Software or hardware design models
      • Software source code
      • Test cases, procedures, and results
      • Plans:
        • Software Development Plan
        • Quality Assurance Plan
        • Software Verification Plan
        • Verification and Validation Test Plan
        • Configuration Management Plan

      The best way to inject quality into work products is through careful engineering and demonstration and test via execution or simulation. For many of these work products, execution isn't feasible, so reviews are a way of assessing the quality of the work product. In addition, many work products must conform to guidelines, standards or templates and this is most easily demonstrated through a review process. Lastly, reviews also serve to disseminate information to different stakeholders about the structure, content, and quality of a work product.

      Steps
      Determine the purpose of the review

      Real models are too large to be reviewed in their entirety in a single review. Each review must have a specific purpose or intent, and subsequently examine only the subset of the model relevant to that purpose. Some common review intents are to perform the following actions:

      • evaluate and understand systems engineering handoff models
      • evaluate the use cases and their details
      • evaluate object analysis (platform-independent) models
      • evaluate design (platform-specific) models
      • evaluate the adequacy and coverage of a test plan and its details
      • evaluate the result of the actions taken after a previous review
      Prepare material for review

      In this step, the material is gathered together for a review. There are almost always multiple artifacts that are reviewed together. For example, a review of a use case model usually includes the following artifacts:

      • Relevant use case diagrams
      • Corresponding requirement text
      • Use case sequence diagrams
      • Use case state machines
      • QoS constraints
      • Use case descriptions
      • List of actions from previous reviews (if applicable)

      For the review of an object analysis model, common artifacts include:

      • For each use case in the object analysis model
        • Class diagram(s) showing the collaboration of elements realizing the use case
        • Original use case sequence diagrams
        • Elaborated sequence diagrams showing the interaction
        • State machine diagrams for stateful classes
        • Activity diagrams for complex algorithms
        • List of actions from previous reviews (if applicable)
      Disseminate materials to reviewers
      The materials should be in the hands of the reviewers no less than 2 nor more than 14 days prior to the review.
      Schedule review

      The room for the review - along with any special equipment required (e.g. projector, white boards, internet connection, computer, modeling tool availability, etc) - is scheduled and invitations issued to the reviewers.

      Reviewers inspect material individually
      The time to read the materials is not within the review meeting itself! The reviewers should come to the meeting with comments and issues they want to address.
      Collectively discuss review comments
      Each issue or comment from a review should be heard and addressed by the product owner. If it is determined to be an issue, the issue is specified on the list of actions to be resolved.
      Perform actions
      As a result of the list of actions identified in the review, the product owner must modify the model or in some way address each of the issues.  Then, if the changes are not trivial, a following meeting is planned (start back with the first step).
      Key Considerations

      Reviews are very expensive to perform but, when performed well, are highly valuable. It is crucial that they be performed effectively. Identified issues should be solved outside of the meeting. Subsequent reviews need only go over the lists of actions from the meeting to ensure each issue has been appropriately addressed. This means that the list of actions is an essential artifact for effective reviews.

      It is crucial to understand that reviews are not a replacement for good modeling in the first place. One of the basic requirements prior to reviewing is that the model executes properly. In a PIM review process, this means that it is debugged and can replicate (via execution) the use case scenarios. In a PSM review process, this means that the model has passed formal unit testing. You should NEVER find a compilation problem in a review meeting; it is far cheaper to have the product owner find that on their own prior to planning the review rather than find it in a room full of people. The intent of the review should be to evaluate an set of already high-quality artifacts.

      It is also very important to stay focused during the review. The meeting leader needs to rule with a (silk covered) iron fist. Reviews are very expensive so the object is to perform the intent of the review as efficiently as possible.

      Note: subsequent reviews of an already-reviewed work product should focus exclusively on the list of actions from the previous review.