Architectural Risk Burndown
This metric tracks the remaining level of architectural risk for a project across iterations.
Main Description

Purpose

Risk is a key driver of architecture and design activities. In order to deliver a feasible architecture, the team must identify, analyze, communicate, and mitigate risk throughout the project lifecycle. Monitoring the progress of risk reduction in a burndown chart provides several benefits:

  • Drives iteration plans and design activities
  • Illustrates progress made in early iterations at addressing architectural issues (when velocity may be low from not delivering more functionality)
  • Provides an easily understood summary of architectural risk that could impact project success. New and escalating risks are visible.
  • Reduces the likelihood of focusing on the wrong issues early in the lifecycle that can result in large surprises at the end of the project. Encourages ongoing and active mitigation efforts and increased communication.
  • Useful for explaining the team's confidence in their estimates based on remaining uncertainty. Helps describe the predictability of the project based on remaining risk.
  • Helps the team understand the effectiveness of their architectural risk mitigation activities

Definition

To calculate remaining risk:

  1. Assign the following values to each identified architectural risk (still outstanding):
  • Likelihood of occurrence - defined as a percentage (using a standard scale such as 20%, 40%, 60%, 80%)
  • Estimated Cost of Impact - defined in hours or days

2. Calculate a score for each risk:

  • Risk Score = Likelihood of occurrence * Estimated Cost of Impact

3. Determine Overall Architectural Risk

  • Total Risk Score = the sum of all individual Risk Score values

A good way to monitor Architectural Risk over time is to use a trend line in a burndown chart. The Total Risk Score is plotted on the Y axis, and the iterations are on the X axis.


Analysis

As outstanding risks are tracked, trends are identified to help the team understand the predictability of their project and overall risk status in terms of the architecture. If sufficient burndown of risk isn't occurring the team can take appropriate corrective action.

Architectural Risk Burndown is used as a capability improvement metric. It helps a team and middle management (project manager, product owner) monitor improvements made during the project lifecycle in adopting the Evolutionary Architecture and Iterative Development practices.

Operational Executives can also use this metric to monitor systemic improvement in reducing time to delivery, improving predictability, increasing value, and improving quality across the organization by adopting Evolutionary Architecture and Iterative Development.

Expected trend - Ideally, risk is decreasing over time through active efforts to drive open risks to closure as the team learns more about the problem domain and the architecture. The team is reducing uncertainty in each iteration, focusing first on risks with the highest likelihood of impact and cost, and is providing better estimates, resulting in a more predictable project. Architectural risks are discovered during the elaboration phase and reduced by 80% before moving into the construction phase.

Flat or rising near the middle of lifecycle - This trend indicates that the team isn't actively working to address risk and there is too much uncertainty in the project. The team should base their iteration planning around the architecturally significant and high risk use cases and non-functional requirements to drive down the outstanding risk. Ensure that test coverage of these requirements is sufficient. Reassess architectural risk during each iteration retrospective and as needed during daily meetings to ensure that high risk items are communicated. Confirm that the architecture is sufficiently captured and communicated so that the team can continue to evolve and deliver it. Provide coaching in architectural risk mitigation techniques. Use the remaining risk level to determine the level of uncertainty in project estimates. If the number of architectural risks remains high through the planned end of elaboration, the team should consider extending the elaboration phase to address those risks.

Fast incline near the end of the lifecycle - This trend can occur when the team did not actively work to identify risks from the earliest iterations (when the impacts of change are least costly) and is presented with surprises at the end of the lifecycle. This can result from waiting too long to deliver working software that proves the architecture. The team should prioritize work items in early iterations that represent architecturally significant use cases, and continue to focus on work items based on coverage and criticality. It can also occur when there has been insufficient test coverage to confirm that the architecture addresses the major aspects of the system, even when there is no expected risk for particular areas. If the team determines that the schedule and cost will be impacted as a result of required work to mitigate late-breaking, newly identified risks, they should collaborate with stakeholders to determine if scope should be cut or the release date adjusted. Determine if it is possible to negotiate impacted architecturally significant requirements in order to reduce high impact risk.


Frequency and reporting

The team should evaluate architectural risk at the end of each iteration during a retrospective meeting. Risk Scores are adjusted as needed, and risks that have been addressed are closed so that the team can update the burndown chart. Trends are monitored and analyzed to help prioritize activities in the next iteration planning session. Stakeholders are routinely apprised of current risks and collaborate with the team for impact assessment and mitigation.

Collection and reporting tools

Risks can be captured in  IBM® Rational® Team Concert®, IBM® Rational® Project Conductor®, IBM® Rational® ReqPro® using custom attributes, or in a simple spreadsheet. Generate the burndown report by using charting capabilities in a basic spreadsheet tool.

Assumptions and prerequisites

  • Time-boxed iterations are scheduled, and iteration length is consistent
  • Risks are captured, and the team collaborates with stakeholders to determine likelihood and impact.

Pitfalls, advice, and countermeasures

  • If risk is not managed consistently throughout the lifecycle, the team will likely begin to act in a reactive, crisis mode as risks emerge. A consistent, repeatable risk mitigation process enables better project predictability and higher levels of trust by project stakeholders.
  • The need to identify and monitor risk increases with the complexity of the system.
  • The effort to reduce uncertainty has an associated cost. The team must balance the cost of this effort against the potential impact of not addressing the risk. Not all candidate risks are actively managed. Mitigation effort and degree of monitoring depend on the likelihood of impact and the risk tolerance for the project.
  • Use the Test Coverage of Requirements metric as a countermeasure for this metric. The team may report low architectural risk, but the system may not have been tested sufficiently to prove the architecture.