Guideline: Preparing Assessment Findings and Recommendations
This guideline helps you consolidate and analyze the information gathered during an organizational assessment and formulate recommendations based on those findings.
Relationships
Related Elements
Main Description

Analyze results

Analyze the collected information in preparation for formalizing the findings. The general steps are:

  • Analyze the gathered data to look for potential problems/issues and articulate the implications of the problems in the context of the organization's business model. Make an effort to look beyond symptoms to identify the root cause of the problems/issues you find. In performing this analysis, one may find ambiguous data among the interviews and reviewed artifacts. Identify where there are ambiguities and schedule another interview or artifact review to resolve the ambiguity. 
  • Identify the recommendations and sort them by Short-term, Medium-term and Long-term actions or High, Medium and Low in terms of impact on business results.
  • During analysis, keep in mind that first impressions can be wrong. It is easy to see symptoms first rather than the underlying root causes. For example, in doctor-patient relationships, doctors who continually treat symptoms rather then the root cause issues lose the confidence of patients (and sometimes even lose the patients themselves!).

During the interviews, you may have heard many statements that indicate symptoms of underlying problem, such as "We're always late getting products out the door" or "Our products don't meet user expectations." You should have probed beneath these statements to find out why these problems occur, from the interviewee's perspective; if you didn't, you might have to go back and ask during analysis.

Equally important, however, is understanding the business impact of the problems discovered during the interviews, as this critical to prioritizing your recommendations. For example, during the interviews you may have captured statements like, "Our problem is that we have no change management process." The lack of an effective change management process is actually a root cause to many problems; therefore, in order to appropriately prioritize the problem, you need to know what sort of impact it has on the business-preferably, a quantifiable impact. It might be that the interviewee felt the delivered product or application doesn't meet expectations, or it could be that the projects are not able to deliver on time. If you didn't get this information during the interview, you might have to go back and ask for it during analysis.

The analysis shouldn't only focus on finding the problems. This can turn out to be a rather negative exercise. There are three pieces of information that are equally important to capture in the findings and recommendations:

  • What sense of urgency to change exists in the organization? And is this something that is understood and shared across all levels in the organization?
  • What things are working well in the development organization? Understanding this is critical, since a change effort that doesn't respect the things that are working may lose credibility and be perceived as doing more harm than good.
  • What behaviors, structures, capabilities, and/or infrastructure represent root causes to why the organization isn't performing well enough? Note that this is not always just about technical skills, processes, or development tools. Many times the structure of the organization, its culture, and history can have an equally strong influence. Although the solutions we provide focus on the former, the latter needs to be recognized in the analysis where appropriate.

During analysis, be careful not to make too much of a single data point, as it could be an aberration and not reflective of the overall situation. It is helpful to think of one data point as an observation, but a second data point is needed to corroborate before one begins to have a finding. As you analyze all the information you have collected, look for these correlated data points and patterns that may be present in the data and that may provide stronger evidence for your conclusions. If you need to go back and talk to someone to corroborate what seems to be an important data point, it is usually worth the effort to do so.

Findings are identified by comparing current processes to some standard or known best practices (e.g. the IBM Practice Library, or CMMI for Development).


Prepare findings and recommendations

Once you have reviewed the process assets, available measurement data, interview notes, workshop outputs and example work products, it is time to prepare the assessment findings and recommendations. Here are some key actions to take as you prepare your recommendations:

  • Review assessment findings with stakeholders and key people to ensure correctness.
  • Identify any areas that require further research (e.g. a referenced process website or further key work products that were not originally inspected or available)
  • Rank the findings and recommendations by their business impact and provide information on specific benefit gained by implementing the recommendations
  • Ensure that the findings and recommendations fulfill the original objectives of the assessment and are within its scope.
  • Provide adequate supporting evidence in the detailed assessment report.

Make sure to include both organizational strengths and areas to improve in the context of the assessment report (usually the five best practices). You should include detailed analysis using specific examples derived from the interview sessions (while leaving out specific names of individuals who may have provided the information); this is necessary to create a credible report.

Avoid expressing findings in the manner of recommendations. Findings represent a current "state". For instance, a finding of "there is no CCB for approving changes" is almost a de facto recommendation; whereas, the real finding may be "changes to the code are made on an ad hoc basis using informal communication mechanisms." With the first, stakeholders may focus first on what it takes to set up a CCB rather then gain an appreciation of the business impacts of their current practice. With the second, the stakeholders more readily accept the finding and the discussion quickly moves to the business risks and impacts.


Validate assessment findings

Once the assessment team has compiled the initial findings and recommendations, they should validate the findings with stakeholders to make sure that that the findings accurately reflect the organization's situation. Solicit feedback you receive and update your findings as appropriate. This step is very important, as it not only gives your stakeholders an opportunity to ask questions and express concerns, but also gives you an opportunity to validate the findings, correct any misconceptions or errors, and to ensure that the manner in which you are presenting the findings will be acceptable to the larger audience of stakeholders and management.

Prepare the deployment roadmap


The roadmap (along with implementation strategy and plan) should include a complete solution composed of process assets, tools, needed training/ mentoring.. Once you have the first draft of the plan, you should collaboratively refine it to arrive at a final version. Be sure to:

  • Accurately summarize the key assessment findings into an executive summary.
  • Present findings in roadmap format proposing incremental improvements. In most cases, you will not replace the software development process currently in practice in the organization. You will implement the new development process step-by-step, focusing on the more critical and important areas first. Some of the current software development process may even continue to exist for some time, perhaps forever. Outline:
    • what activities are to be performed
    • benefits of process and tool improvements
    • include schedules, deliverables, estimated costs, risks, and constraints.