Gather information
The objective is to identify how the organization current approaches software delivery. Information can be gathered
through several methods:
-
Analyzing the outcome of previous assessments and projects
-
Interviews with stakeholders and key people
-
Reviews of current software delivery process assets and project artifacts
Conduct a kickoff meeting
Kick off the assessment with stakeholders and key people by giving an overview of the assessment goals and activities.
Confirm the scope of the assessment and ask key representatives to give an overview of current software delivery
processes. This meeting should not be overly formal or take too much time. A focused 1-2 hour session is usually
sufficient. Make sure to request feedback during and after the session about the approach for conducting the
assessment.
Conduct interviews
The objective of conducting interviews is to gather detailed information about how teams across the organization
performs software delivery activities, so that you can analyze strengths and weaknesses.
Preparing for the interviews
Before you begin the interviews, confirm the following:
-
Interview schedule
-
Names of those who will conduct each interview and note takers
-
List of interview questions
-
Tools used to record responses, such as:
-
-
Voice recorders
-
Mind mapping tools e.g. MindManager, Freemind
-
Microsoft® Word® or Excel®
-
IBM® Rational® Software Architect, Rational® RequisitePro®, SoDa
-
How will you structure the results report (e.g. by practice, discipline, priority)
Probe for actual processes
Remember that documented processes may not actually be implemented. Those in use on real projects may be quite
different. As you conduct interviews with project teams, confirm that you review real project artifacts and discuss the
actual process followed to create them.
During each interview, confirm the following the interviewee:
-
The interviewee's roles and responsibilities.
-
Other roles they interact with to perform their responsibilities.
-
Processes performed.
-
Their opinion on the top 5 problems that currently prevent them to perform their job most effectively.
-
Improvements they think are most worthwhile.
-
Processes they perform.
-
What is currently working well and should not be changed.
Follow the list of specific questions prepared in advance that are based on the particular role of the
interviewee. However, be prepared to "go off-script" if the interview leads in a direction that you did not anticipate
but is providing useful information.
In interviewing people who are in roles that are above the project level, devote more time within the interview to
better understand business drivers and the basic business model. Managers can have differing viewpoints, and executive
visions are not always implemented cohesively throughout an organization. You can use the interview questions template
to guide you in what questions to ask.
Put interviewees at ease
Some interviewees may see an assessment interview as a threat. They may not be following the prescribed process
(many times, for good reason). To be able to capture this type of insight, the interviewees must be comfortable with
the interview and be confident that their work habits and views will not have an adverse affect on them.
With this in mind, prepare an interview environment that helps put them at ease:
-
The interview setting should be in a private area, such as a conference room that has been reserved for the
interviews.
-
Explain that the purpose of the interview is to assess the current state of affairs and that specific
recommendation for improvement will be presented as a result of the assessment. Any interviewees who were not at
the kickoff meeting may not have a clue as to why managers asked them to meet with you.
-
Assure the interviewees that no one will be identified as the source of any specific comments.
-
Assure the interviewees that there may be questions that they may not be able to answer, and that is OK. They also
may not have data at hand to back up their answers, and that is also OK, but tell them that you will need to get
the data later.
-
Explain that you will be taking notes, but that these are for the assessment purposes only and will not be shared
outside of the assessment team.
-
Let the interviewees know that they will be given an opportunity at the end of the interviews to ask questions.
-
Explain the type and form of feedback that they should expect to receive as a result of their participation
(Findings Presentation, Assessment Report).
Probing for business impact
During interviews, it is easy to see symptoms first rather than the underlying root causes. It is important
to uncover the root causes of the problems, inhibitors to success, and the business impact of those problems to be
able to prepare credible and compelling findings and recommendations.
Sometimes, you will hear statements that indicate a symptom of an underlying problem, such as "We're always late
getting our products out the door" or "Our products don't meet user expectations." You need to probe beneath those
statements to find out why these problems occur, at least from the interviewee's perspective. Try to corroborate what
you have found by asking other people, in subsequent interviews, who might see the same problem from another
perspective.
Other times, you will hear statements that directly express a root cause, such as, "Our problem is that we have no
change management process." The lack of an effective change management process is actually a root cause to many
problems; therefore, to appropriately prioritize the problem, you need to know what sort of impact it has on the
business. It might be that the interviewee felt the delivered product or application didn't meet expectations, or it
could be that the project teams are not able to deliver on time. If you can get the person to quantify the impact of
the problem, all the better.
Be aware that when you hear someone state that something is a problem, it might not be accurate. They may have a change
process, but not be aware of it, not agree with it, or simply not be part of it. Take any suggested solutions
seriously, but be sure to verify the information with many interviewees.
You can also probe the interviewees for their opinions on who in the organization is most affected by this problem.
That can lead to additional stakeholders who were not on the original list of interviewees but might be able to provide
valuable insight.
Closing questions
When closing the interviews, be sure to give the interviewees a chance to express or summarize their opinions of
what changes should and should not be made. Answers to these questions can contribute directly to your key findings in
terms of "areas for improvement" and "areas of strength." You can ask, for example:
-
From your vantage point, what changes, if implemented, would have the most significant positive impact on your
area?
-
As the organization moves forward, what areas of strength would you like to see preserved?
Review development environment
Project artifacts
Review multiple representative samples of each key artifact if at all possible. It is best to not only determine
the state, organization, coherence, and clarity of process definition, but also the state, quality, consistency, and
compliance of the related templates and artifacts. This can provide a more objective structure for point-based
evaluations according to dimensions derived from these characteristics.
In collecting types of artifacts for review, consider the following categories:
-
Work products
-
Process or methodology definitions and procedural guidelines
-
Planning and management artifacts
-
Policy and governance documents
-
Measurements and metrics
-
Status and reporting artifacts
-
Training materials and user documentation
As you review project artifacts, look at the structure of the artifact and whether it conforms, at least in spirit, to
the defined process. Try to get answers to these questions:
-
What is the process for controlling or tracking changes to artifacts? What is the change control board
for project artifacts?
-
Are the artifacts are managed in a digital repository or available only on paper?
-
Is each artifact used to add value to their project or just to put a "check in the box" to satisfy the process
requirements?
-
Are the artifacts are a by-product of development or the focus of the development? For example, are teams
spending a lot of time writing documents rather than building the application?
-
Are the artifacts created as an integral part of the process or as "after the fact" documentation of the process?
-
Can teams automatically trace artifacts from one development phase to another (requirements mapped to test
cases and so forth)?
-
Are the processes and artifacts consistent across projects and departments? For example, does one project have a
very different requirement set than another project?
Just scan the artifacts. You don't have time for an in-depth review, and that is not the purpose of the
assessment.
Consolidate the interview results
The assessment team analyzes the interview data and supplementary information to produce the initial findings and
recommendations. Ambiguities in the findings may necessitate follow-up interviews or access to artifacts. Two key
activities are:
-
Analyze and consolidate interview results
-
Create assessment findings and validate in order to gain consensus.
Identify any areas that require further research (e.g. a referenced process website or further key work products that
were not originally inspected or available).
Have sufficient domain expertise available in the assessment team to perform the best practice gap analysis when
preparing interview findings.
|