Introduction
Development organization assessments are performed in order to understand an organization's current software
development capabilities. The assessment team learns how the organization currently approaches software
development and delivery and identifies current process weaknesses and strengths. The assessment team then makes
improvement recommendations that are aligned with the organization's operational objectives and validates them with
stakeholders.
Factors to consider
When performing an assessment of the organization's current software capabilities, consider the following factors which
should shape the character of the organization's processes and tools usage, and their required tailoring guidelines.
Operational Objectives
A successful assessment requires understanding the organization's operational objectives. These objectives are
internally-focused, concentrating on operations within the company and their ability to perform the most critical
functions to meet business obligations. Operational objectives typically target these criteria:
-
Productivity or efficiency
-
Time to delivery
-
Quality
-
Predictability
-
Value and customer satisfaction
Organizational processes should be defined to align with the organization's operational objectives. For example, if
quality is a priority, confirm that the organization has sufficient practice and tool support to prevent and detect
defects.
Prior to beginning the assessment, review and validate the operational objectives.
Business context
There are different types of business contexts that affect the required level of ceremony, the level of formality,
and the rigidity of the process. The more stakeholders (e.g. buyers, customers, subcontractors, regulatory bodies)
involved, the more likely projects will need to produce formal evidence, such as documents, reports, and prototypes, at
major project milestones. Examples to consider:
-
Contract work where the developer produces software to a given customer specification and for this customer only.
-
Speculative or commercial development where the developer produces and covers the cost of putting the software on
the market.
-
Internal projects where customer and developer are in the same organization.
Size of typical software development efforts
The typical effort's size will also affect the level of ceremony, the level of formality, and the rigidity of the
process. The larger the project, the larger the development team and, regardless of the business context, the more
formality and visibility the various teams and management need to have in requirements, interfaces, and progress
indicators. Communication issues on large projects are further aggravated by geographically dispersed teams.
The size of software development efforts can be defined by certain metrics such as Source Lines of Code
(SLOC), Delivered Source Instructions or Functions Points, number of person-months or merely the cost.
Degree of novelty
The degree of novelty is based on what has preceded a given software effort within the development
organization and, in particular, whether the development is in a second or subsequent cycle. This includes the maturity
of the organization and its process, its assets, its current skill set, and issues such as assembling and training a
team, acquiring tools, and other resources.
A given project's degree of novelty affects the process in a completely different way. For a new type of project, the
first of its kind, the inception and elaboration phases will be longer, and may span several iterations. Also, more
emphasis will be put on eliciting and capturing requirements, on use-case modeling, on architecture, and on mitigating
risk. For a project that is an evolution cycle from a previous system, change management is more crucial and
incorporating legacy code poses some technical challenges.
Novelty is not only relative to the system being developed, it's also relative to the maturity of the performing
organization because introducing new techniques or tools affects the process.
Type of applications
There are different types of applications (e.g. embedded real-time systems, distributed information systems,
telecom systems, Computer-Aided Software Engineering (CASE) tools). The type of applications delivered by an
organization will affect its processes, especially with respect to specific constraints the domain may impose
on the development such as safety, performance, internationalization, memory constraints, and so forth.
The type of application may affect the process if the application is mission-critical; for example, the flight-control
system in an airplane. A mission-critical system requires a higher level of ceremony in general, both to trace
requirements and to assure the quality of the product. A mission-critical application also requires that more resources
are spent on testing.
The type of development, or the target domain, bring in process issues such as:
-
Techniques and tools to support specific activities; for example, automatic code generation for finite-state
machines.
-
Certification procedures; for example, for medical instrumentation.
-
Compliance to standards; for example, for accounting or fiscal issues, and for telecommunication equipment.
Type of development
There are various types of development, such as:
-
Contract work where you develop a product for a specific customer. You have more stakeholders to manage and
negotiate with hen you perform contract work. There is often a need for more formal external artifacts because the
customer, or representatives, want to monitor progress and be kept informed. Make sure that the artifacts the
customer reviews are easy to understand. Sometimes, there is a need to have a milestone where the project can
offer a fixed-price on the rest of the project. In that case, you may need to add a new milestone or adjust the
existing milestones. In other cases, you may have to adjust to the lifecycle model the customer is using with other
milestones and phases.
-
Speculative development where you develop a product for a mass-market. In speculative development, a marketing
(product) manager, within the organization itself, acts as the customer. Time-to-delivery is often more important
than the functionality in speculative development and there is less need for formal reviews.
-
Internal development where you develop a product that is delivered to another department within the company. It may
be acceptable to be less formal when describing artifacts because they will be reviewed by peers.
-
Pre-studies where you do not normally develop a product. The purpose of a pre-study project is to find out whether
it is possible to build a product. A pre-study project doesn't have the same milestones as a typical
project.
Technical and managerial complexity
Different types of systems, and their projects, can be classified in terms of the technical complexity of the system
and the managerial complexity. For example, a typical small business spreadsheet application is often of low technical
complexity and is easy to manage. The other extreme is a typical weapon system project, which is often both technically
complex, and complex to manage.
Usually increasing system size, project duration or business context also increases the managerial complexity.
Increasing the novelty, in either the problem domain or the solution space, increases the technical complexity. There
is an interaction between managerial and technical complexity as well, as many large projects are also technically
complex. This results in:
-
Increased managerial complexity that leads to more ceremony, including more formal reviews and milestones, and more
artifacts.
-
Increased technical complexity that leads to the introduction of specific techniques, roles and tools, and,
therefore, more activities.
Problems and root causes
An important aspect of understanding the software development organization is to understand the problems in the
existing software development process. This influences those areas of the process you will concentrate on in the
beginning of the process implementation. It is important to note that, if there is no established way of working
in the organization, it may be pointless to find problems. Instead, you may need to identify the root causes of the
problems. To eliminate the problems, you will tackle the root causes by improving the process, introducing tools to
automate the process, and training people.
Examples of common problems:
-
Inability to manage scope-the organization routinely tries to do more than they actually do in the end.
-
Inability to capture requirements-they have difficulty specifying requirements.
-
Inability to manage changing requirements.
-
Inability to manage requirements-requirements do not make it to the final product.
-
Inability to estimate-they are routinely too optimistic about their ability to deliver on schedule.
-
Design deficiency-they are good at meeting requirements, yet poor at designing systems. What kinds of design
problems do they have? Are the systems difficult to maintain and enhance? Do they have performance problems,
usability problems, capacity problems, and so on?
-
Inability to produce quality products-the product has too many defects which may be due to lack of testing, but
usually is also related to an inability to capture and manage requirements, as well as design deficiency.
-
Unacceptable software performance.
-
Low usability.
-
Colliding developers-there is a lack of control over ownership and configuration management, so that developers
make conflicting changes and work is lost.
-
Late discovery of problems.
-
Trouble going from use cases to design.
Examples of root causes
A problem is often a symptom that something is wrong. You need to identify the root causes of the problems. The
following are examples of some common root causes:
-
Insufficient requirements management
-
Ambiguous and imprecise communications
-
Brittle architectures
-
Overwhelming complexity
-
Undetected inconsistencies among requirements, designs, and implementations
-
Insufficient testing
-
Subjective project status assessment
-
Delayed risk reduction due to waterfall development
-
Uncontrolled change propagation
-
Insufficient automation
-
No systematic way to build user interfaces
-
No way to go from use cases to a design
Advice for conducting the assessment
|