Analyze results
Analyze the collected information in preparation for formalizing the findings. The general steps are:
-
Analyze the gathered data to look for potential problems/issues and articulate the implications of the problems in
the context of the organization's business model. Make an effort to look beyond symptoms to identify the root cause
of the problems/issues you find. In performing this analysis, one may find ambiguous data among the interviews and
reviewed artifacts. Identify where there are ambiguities and schedule another interview or artifact review to
resolve the ambiguity.
-
Identify the recommendations and sort them by Short-term, Medium-term and Long-term actions or High, Medium and Low
in terms of impact on business results.
-
During analysis, keep in mind that first impressions can be wrong. It is easy to see symptoms first rather than the
underlying root causes. For example, in doctor-patient relationships, doctors who continually treat symptoms rather
then the root cause issues lose the confidence of patients (and sometimes even lose the patients themselves!).
During the interviews, you may have heard many statements that indicate symptoms of underlying problem, such as "We're
always late getting products out the door" or "Our products don't meet user expectations." You should have probed
beneath these statements to find out why these problems occur, from the interviewee's perspective; if you didn't, you
might have to go back and ask during analysis.
Equally important, however, is understanding the business impact of the problems discovered during the interviews, as
this critical to prioritizing your recommendations. For example, during the interviews you may have captured statements
like, "Our problem is that we have no change management process." The lack of an effective change management process is
actually a root cause to many problems; therefore, in order to appropriately prioritize the problem, you need to know
what sort of impact it has on the business-preferably, a quantifiable impact. It might be that the interviewee felt the
delivered product or application doesn't meet expectations, or it could be that the projects are not able to deliver on
time. If you didn't get this information during the interview, you might have to go back and ask for it during
analysis.
The analysis shouldn't only focus on finding the problems. This can turn out to be a rather negative exercise. There
are three pieces of information that are equally important to capture in the findings and recommendations:
-
What sense of urgency to change exists in the organization? And is this something that is understood and shared
across all levels in the organization?
-
What things are working well in the development organization? Understanding this is critical, since a change effort
that doesn't respect the things that are working may lose credibility and be perceived as doing more harm than
good.
-
What behaviors, structures, capabilities, and/or infrastructure represent root causes to why the organization isn't
performing well enough? Note that this is not always just about technical skills, processes, or development tools.
Many times the structure of the organization, its culture, and history can have an equally strong influence.
Although the solutions we provide focus on the former, the latter needs to be recognized in the analysis where
appropriate.
During analysis, be careful not to make too much of a single data point, as it could be an aberration and not
reflective of the overall situation. It is helpful to think of one data point as an observation, but a second data
point is needed to corroborate before one begins to have a finding. As you analyze all the information you have
collected, look for these correlated data points and patterns that may be present in the data and that may provide
stronger evidence for your conclusions. If you need to go back and talk to someone to corroborate what seems to be an
important data point, it is usually worth the effort to do so.
Findings are identified by comparing current processes to some standard or known best practices (e.g. the IBM Practice
Library, or CMMI for Development).
Prepare findings and recommendations
Once you have reviewed the process assets, available measurement data, interview notes, workshop outputs and example
work products, it is time to prepare the assessment findings and recommendations. Here are some key actions to take as
you prepare your recommendations:
-
Review assessment findings with stakeholders and key people to ensure correctness.
-
Identify any areas that require further research (e.g. a referenced process website or further key work products
that were not originally inspected or available)
-
Rank the findings and recommendations by their business impact and provide information on specific benefit gained
by implementing the recommendations
-
Ensure that the findings and recommendations fulfill the original objectives of the assessment and are within its
scope.
-
Provide adequate supporting evidence in the detailed assessment report.
Make sure to include both organizational strengths and areas to improve in the context of the assessment
report (usually the five best practices). You should include detailed analysis using specific examples derived
from the interview sessions (while leaving out specific names of individuals who may have provided the information);
this is necessary to create a credible report.
Avoid expressing findings in the manner of recommendations. Findings represent a current "state". For instance, a
finding of "there is no CCB for approving changes" is almost a de facto recommendation; whereas, the real finding may
be "changes to the code are made on an ad hoc basis using informal communication mechanisms." With the
first, stakeholders may focus first on what it takes to set up a CCB rather then gain an appreciation of the
business impacts of their current practice. With the second, the stakeholders more readily accept the finding and the
discussion quickly moves to the business risks and impacts.
Validate assessment findings
Once the assessment team has compiled the initial findings and recommendations, they should validate the findings with
stakeholders to make sure that that the findings accurately reflect the organization's situation. Solicit feedback you
receive and update your findings as appropriate. This step is very important, as it not only gives your
stakeholders an opportunity to ask questions and express concerns, but also gives you an opportunity to validate the
findings, correct any misconceptions or errors, and to ensure that the manner in which you are presenting the findings
will be acceptable to the larger audience of stakeholders and management.
Prepare the deployment roadmap
The roadmap (along with implementation strategy and plan) should include a complete solution composed of process
assets, tools, needed training/ mentoring.. Once you have the first draft of the plan, you should collaboratively
refine it to arrive at a final version. Be sure to:
-
Accurately summarize the key assessment findings into an executive summary.
-
Present findings in roadmap format proposing incremental improvements. In most cases, you will not replace the
software development process currently in practice in the organization. You will implement the new development
process step-by-step, focusing on the more critical and important areas first. Some of the current software
development process may even continue to exist for some time, perhaps forever. Outline:
-
what activities are to be performed
-
benefits of process and tool improvements
-
include schedules, deliverables, estimated costs, risks, and constraints.
|