Assess the Tools Configuration and Infrastructure
The assessment of the infrastructure for the measurement system is the easiest element to evaluate. The tools and their
configuration either work or they do not. They either efficiently support automatic collections or they do not. Take
corrective action immediately to resolve any issues associated with infrastructure. Languishing infrastructure issues
(connection, performance, down time, and so forth) kill any attempt at establishing a measurement system.
Organizations have much fewer problems with the infrastructure if they deploy the measurement system to the
organization incrementally and pilot the use of each increment before moving the contents out to the broader
population. Some metrics will not be affected. However, those showing trends over time are extremely vulnerable to lack
of confidence when there are issues that arise that could have been resolved on a small scale early.
The longer it takes to show growth toward organizational and business objectives, the less likely the measurement
system will continue.
Assessing the configuration and infrastructure should include items such as:
-
Issues with the hardware and software for the measurement system
-
Issues with hardware down time
-
Issues with availability of metric reports
-
Correctness of the derived measures
-
Usability of the metric reports
-
Timeliness of the metric reports
|
Assess the Measurement Process
The measurement process (who, what, how, and when) needs to be evaluated regularly. As businesses and IT organizations
change, so should the content of their multi-tiered measurement system that supports them. As the business changes, or
as IT changes, the focus may shift from one set of organizational objectives to another. To measure the impact of
the changes associated with these new objectives, an additional or a different set of measures may need to be used.
A note of caution:
Changing a measurement system too much, too soon or too often will send the message that the newest measurement set is
simply another measure of the month. Before deploying too much change too soon, it is advisable that the
current set of measures have demonstrated achievement of the goals set for them (for example, institutionalizing a set
of practices and techniques, deployment of a suite of tools across the entire organization, and so on).
|
Assess Team Usage Models
Assess the understanding of those who are using the deployed metrics. This includes understanding and proper use at all
levels of the IT organization. The effectiveness of a deployed performance system increases with its use; nonetheless,
many attempts at sustaining a measurement system have failed due to the lack of use and commitment.
Any metric not used should be removed. Therefore, project teams and management must use the metrics for what they were
intended. The purpose of a performance measurement system is to provide a mechanism for reasoning about the
effectiveness of a change effort and the efficiency of tools and people towards achievement of business goals. Periodic
assessments should occur if for no other reason than it sends a clear statement from management that the measurement
system is important. This will not allow it to become forgotten and useless.
Not having a vibrant multi-tiered measurement system gives rise to the questions of value that are directed internally
toward organizations as a whole and at people, processes, and tools in particular. To create and maintain a vibrant
measurement system requires commitment by everyone at all levels in IT. A programmer or a technical writer may not be
that interested in a metric about meeting milestones or a metric related to project financials, but they should be made
aware of what changes they may need to make or that the organization needs to make to create a flexible organization
that provides value to the business. It is the business that provides the work environment, and it is the role of IT to
support the business in meeting its objectives.
|
|