Task: Assess the Organization
Evaluate the organization's software development capabilities and recommend improvements.
Disciplines: Software Capability Improvement
Purpose

Assess the organization's software processes and tools usage in order to make recommendations for improvement against operational objectives. This task is performed so that you can:

  • understand your current state and drivers for improvements. Establish a baseline for the planned ongoing measures of success and operational performance. 
  • identify those areas that need to be improved first, starting with the areas that have the greatest need and the best potential for improvement. 
  • explain to sponsors why you need to change the process, tools, and people.
  • create motivation and a common understanding among the people in the organization regarding who will be directly and indirectly affected by the changes.
  • create a roadmap of capability improvements that aligns with your organization's business objectives.
Relationships
RolesPrimary Performer: Additional Performers:
InputsMandatory:
  • None
Optional:
    Outputs
      Main Description

      To improve software development capabilities it is important to understand problem areas and potential areas for improvement, as well as information about external issues such as competitors and trends in the market. When this task is complete, you should understand:

      • the current state of the software development organization
      • the kinds of people involved and their levels of competence, skills, and motivation
      • the tools currently used by the organization to support software development
      • the current software engineering process and how it is described
      • what improvements are needed in order to meet operational objectives

      Steps
      Prepare for the assessment

      Obtain commitment from senior management for the assessment. Their sponsorship is required to provide adequate funding and to ensure managers and staff participate in the assessment activities.

      Confirm that your organization's operational objectives have been identified as these will drive priorities for capability improvement.

      Define the scope of the assessment based on organizational challenges. Assessments can focus on the entire organization or on particular business units or teams. 

      Establish the assessment team. Identify suitable and available staff members, including one to fulfill the role of lead assessor.

      Prepare initial assessment materials:

      • Agenda
      • Communication plan
      • Interview questions
      Identify the stakeholders

      Identify the stakeholders for the process and tools improvement effort.These are individuals or groups with a material interest in the success of the project. Start by identifying  and describing the different types of stakeholders and their responsibilities. For each stakeholder group, identify their objectives and high-level needs to be met by the project. Include stakeholders:

      • Outside the development organization, such as:
        • Customers.
          Who are the customers? What requirements do customers have on the products in terms of time-to-market, features, security, robustness and safety, and complexity?
        • Competitors.
          Who are the competitors? In what areas are the competitors strong? What can you learn from competitors?
        • Other stakeholders.
          Are there any other stakeholders involved? Are suppliers and partners involved? Are relationships with them a problem?
      • Within the development organization, such as:
        • Project members
        • Project managers of completed or ongoing projects
        • Marketing
        • People in other development departments
      Identify key people

      Identify the key people in the organization that can help drive the effort of deploying the development processes and tools. A key person is someone who has one or several of the following characteristics:

      • has the ear of the masses.
      • can act as a mentor.
      • is an expert in some area.
      • is opposed to the process implementation.
      • is responsible for the budget.

      For successful process and tools adoption it is important to have the key people on board to do the following:

      • gathering information during the assessment.
      • helping to tailor the processes and tools configurations.
      • working in pilot projects, then be mentor.
      • harvesting experience from the projects using the processes and tools.

      However, be wary of people who want to discuss process and methodology, instead of developing software.

      Describe the internal organization

      Describe the internal organization, particularly the current roles and teams. Also, look at the relationship between different parts of the development organization. For example, what is the relationship between development and maintenance, and what is the relationship between development and test? How well do the groups coordinate their related activities?

      Identify competencies

      Make a list of relevant competency areas in scope of the assessment such as:

      • Process
        • Business modeling
        • Requirements
        • Graphic user interface design
        • Analysis and design
        • Implementation (e.g. Java, C++, Relational databases)
      • Tools (e.g. IBM Rational RequisitePro, IBM Rational Team Concert)
      • Problem domain
      • Technology (e.g. distributed systems)
      • Administrative and organizational skills
      • Interpersonal and communication skills
      • Organizational knowledge

      For each of these competency areas, assess the knowledge, expertise, and experience of the people in the organization. There are several ways to do this:

      • Ask one or several managers to rank the competencies of individuals or entire teams.
      • Ask the employees themselves to rank their own competencies in each area.

      For each area, estimate the individual's competency. The following is an example of a simple competency scale that can be used:

      • Expert. Can act as mentor to others.
      • Working knowledge. Can work independently.
      • Basic knowledge. May occasionally need help.
      • Novice. May not be familiar with the subject.

      Make a competency profiles of individuals and compile competency profiles for entire teams. It is just as important to understand the competency profile of a team as a whole because no individual alone can have all the necessary competencies.

      Assess attitudes

      Interview people to understand their attitudes toward changing to new technology, new tools, and new process. If people are negative or skeptical toward the change, it is impossible to succeed with the change unless you turn their negative attitudes into positive ones.

      Understand the process culture of your organization. Are teams used to a prescriptive, high-ceremony approach? Or are they of a more agile mindset? Or a mix? 

      Estimate the capacity for change

      Analyze the capacity for change in the organization and among its individuals. When looking at the organization's problems, there may be a tendency to want to fix everything all at once, especially since many of these problems occur together. This is a fatal trap. Organizations, just like individuals, can accommodate change only within a limited range. Different organizations are better prepared for change than others.

      Understand if resource changes will be required to implement and support proposed improvements. What are the associated costs of these changes?

      Characterize the projects and applications

      Describe the characteristics of projects and applications typical for the organization. Look at both finished projects and projects currently running. Collect information by interviewing individuals or entire teams. Study the results produced by projects (e.g. project plans, artifacts produced by the project, process descriptions, project testimonials and final status assessments).

      Collect the following characteristics about each project and its product in order to understand the factors and what effect they will have on how your process descriptions.

      • Size of the software development efforts. What is the size of the product?
      • Degree of novelty. Where, on a scale between "green-field development" and maintenance, is the product?
      • Type of application. Is the application mission-critical? Are there any requirements to follow specific standards?
      • Identify what type of development the organization typically conducts. For example, is it contract work, speculative development, internal development or pre-studies. 
      • Technical complexity. What are the technical problems and challenges in building the product? For example, is the application a distributed system, a safety-critical system or a high-integrity system? Are there any legacy systems that will be reused?
      Assess development process descriptions

      Review current descriptions

      Read existing process descriptions to understand the depth of the existing development process. For each discipline or process area, identify what kinds of description is available for that discipline. For example, find out if there are no descriptions at all, and whether there are examples, guidelines, document templates, activity descriptions, artifact descriptions, and so on. Confirm that processes are well integrated. Identify integration points between processes (e.g. what input and output work products are shared between processes).

      Remember that different teams may be using different processes for the same capability.

      Compare to process standards

      Consider any process definition requirements as you perform your assessment. Is your industry highly regulated, requiring a heavy evidence-based process? Are you trying to achieve a CMMI maturity level that requires proof of performing each process requirement? 

      It is helpful to compare your process against some standard such as the IBM Rational Unified Process, or to an improvement approach such as CMMI for Development. This can help you identify any gaps or areas of insufficient coverage. However, do not implement process improvements just to cover gaps. Confirm there is an organizational need (the process will help you achieve some operational objective).

      Guideline: Considerations for Organizational Process Definition by Process Area describes organizational process definition considerations by process area and can help you identify current gaps in your process and tool coverage.

      Assess consistency

      Confirm whether processes are applied consistently.  Are the process descriptions sufficiently described to be repeatable across projects? Do tailoring guidelines exist so that individual project teams know how to apply the process in their context?

      Conduct interviews

      Make sure to interview your key people and stakeholders as you evaluate process descriptions. Insight from those actually using the process and who understand what problems need to be addressed is critical to the assessment.


      With a good understanding of the existing process, you will:

      • understand what type of process description teams are used to.
      • understand what areas they need most for a new process description.
      • understand what parts of the process description you can continue to use.

      Assess supporting tools

      Identify the tools that are currently being used by the projects. Identify the areas where project members lack tool support to automate tasks, streamline work, or provide evidence of the process.

      Confirm tool configurations are defined with tailoring guidance to meet the needs of individual projects.

      Assess work environment standards

      In addition to common processes and tools, organizations can benefit from common training and work environment standards to support productivity, health, safety, and ergonomic factors.

      Work environment standards can also address guidelines for tailoring and obtaining waivers for deviations from the standards to meet the needs of a particular project.

      Identify opportunities for improvement

      Analyze measurement data to identify problem areas. Roll up project data to the organizational level to measure performance against the thresholds established for operational objectives.

      Assessment workshops are also helpful for identifying improvement opportunities. See Guideline: Conduct an Assessment Workshop.

      Also see Guideline: Identify Opportunities for Software Capability Improvement

      Draw conclusions

      Analyze the results of the collected information and compile a list of areas and issues on which to focus. Issues that should be addressed early usually fall into one or several of the following categories:

      • major problem areas where you can significantly improve the performance of the projects
      • areas where you can make short-term profits and where you can quickly show results
      • areas where an improvement will have high visibility

      Make sure to also compile a list of what is working well for the organization in terms of processes and tools usage as well.

      Validate findings with stakeholders to achieve consensus.

      Make recommendations

      Recommendations for the future are developed as part of the assessment. Each recommendation should be prioritized and describe how the development organization should go forward to implement process and tool improvements. The recommendations could consist of actionable road maps, outlined implementation plans, project plans, tailoring suggestions, and so on.

      Make recommendations based on the strengths and weaknesses identified in the assessment, defining clear goals of what to improve. Ensure that the roadmap for implementation is aligned with the organization's operational objectives. Always pay attention to the reasons why the organization should adopt a new process.

      Agree on measures for success/improvement from process and tool adoption:

      • Select from a common set of measures
      • In addition to objective measures, some "subjective measures" can also be identified
      • Focus on key success factors, ideally defined as part of an analysis of Return-On-Investment (ROI).

      Determine your recommendations based on a recent assessment. Do not wait for months after an assessment to prepare improvement road maps as the capability snapshot taken at that moment in time may no longer apply.

      Present recommendations to stakeholders and key people to achieve consensus and confirm support.  An understanding of the deployment vision and the steps required to implement this vision is critical. This vision must be shared and internalized by all stakeholders.


      Key Considerations

      A successful assessment requires:

      • an identified set of operational objectives against which to measure process performance
      • availability of project and team leads, experienced assessors, and domain subject-matter experts
      • organizational commitment to identify and free-up key staff for the process
      • facilitation of honest and accurate communication about the state of current processes and tools usage

      When providing recommendations for improvement, keep in mind that introducing many changes all at once is rarely successful. Plan a few changes, measuring the results, and then undertaking further change based on those results.

      For changes to be effective, the people who will need to change their behavior (the actual teams developing software)  must be involved in the entire assessment and recommendations effort. 

      Be aware of any potentially conflicting initiatives.

      Assessments are typically performed once per year to ensure that the organization's processes are aligned with its business and operational objectives.

      More Information