Guideline: Goals and Measures
This guideline provides information to help individuals of an organization move from a stated goal to a defined, measurable goal.
Relationships
Main Description

Most organizations have stated goals the business or an IT organization is to work toward. Invariably, the words are ambiguous and difficult to directly trace with quantifiable outcomes. These goals must be decomposed into sub-goals using quantifiable terms which in turn, are measurable.

In any IT organization it is a long way from individual and team behavior and contributions to a business goal. Almost everyone will state that doing such and such should improve what ever business goal is being addressed. But the question is not one of intuition or even logic but one of proof. How does one prove that changing certain aspects of individual or team behavior directly affects one or more stated business goals? The answer lies in the ability to decompose goals into sub-goals until measurable data is reflected in the sub-goal.

For example, a business might chose to emphasize the improvement of customer satisfaction. Let's assume that customers of a product maintained by an IT department are very dissatisfied with the quality that product and the satisfaction rating coming from the customers has been dropping over a period of years. There is much to uncover to understand all of the variances contributing to low customer satisfaction. For a performance measurement example, we will assume that what needs to be uncovered has been or will be uncovered, but our effort is just to define the measures for monitoring progress towards that improvement with the subsequent tasks of creating and configuring the actual infrastructure to implement the measures.

At the beginning, the goal might look something like, "Improve customer satisfaction." To keep from wasting a lot of time, this stated goal needs more clarification in order to establish measures unique to this IT department and their customers. Two useful techniques in refining goals and setting proper context of understanding are to use S.M.A.R.T. (Specific, Measurable, Achievable, Relevant and Time-bound) attributes with the Goal Question Metric (GQM) approach.

The S.M.A.R.T. technique is to define the goal with S.M.A.R.T. attributes. GQM focuses on goal questioning to drive towards data elements that are measurable. Using both approaches aids in decomposition of an ambiguous goal into sub-goals for which metrics can be identified. Using both techniques together may result in changing the focus of the original goal due to better understanding of what is needed by the organization.

Goal Setting with S.M.A.R.T. and GQM

Specific

A goal needs to be specific. The goal of, "Improve customer satisfaction" appears to be specific from the perspective of what to focus on. But there could be any number of methods used to improve customer satisfaction. Which ones are the right ones? Once the right ones are implemented, how is the implementation of the necessary changes progressing? Questions like the following help add clarity around the goal:

  • How will we be sure the satisfaction has actually improved?
  • Why is it necessary to improve customer satisfaction?
  • Improving customer satisfaction results in what?
  • What is causing low ratings of customer satisfaction?
  • How much improvement is needed?

Answers to these kinds of questions help develop the appropriate context to assist an organization with the specificity of the goal. The answers should result in the one goal becoming multiple goals, or sub-goals, focused on the following associated elements:

  • Improve customer satisfaction
  • Increase revenue from product ABC
  • Improve product quality

It is no stretch of the imagination to go from improved product quality to improved customer satisfaction and then for a business to increased revenue for the product. In an attempt to be more specific about a goal this organization now has three goals that are interrelated. But it is still a little bit of a "leap of faith" to get to the "right" set of measures for monitoring the effectiveness of the change effort that is expected to bring about the improvement.

Measurable

To know that a goal has been or will be achieved, it must be measurable. While a specific goal might be to increase the customer satisfaction index, to know that it has been accomplished there must be one or more data elements (attributes) to measure in order to reason about the success of achieving the goal. In order to understand what those attributes are or where to look for them, the following questions are the types of questions to ask:

  • How much improvement is needed in customer satisfaction to be considered successful?
  • How much revenue increase is necessary in order to demonstrate that what ever changes occur at individual and team levels define the criteria of success for those changes?
  • How long is reasonable to reason whether the goal is being met?
  • What data elements are needed to build the measure(s) around?

Restating the previously stated interrelated goals as measurable sub-goals may result in something like the following where x and y in each statement are expressed in quantifiable terms (i.e., attributes than can be measured):

  • Improve the customer satisfaction index from 75 to 90, a 20% improvement.
  • Increase the amount of revenue from product ABC from $1.25 million to $5 million.

Our example is rather straight forward and it may actually take several attempts to get to a sub-goal level that identifies attributes that are directly measurable.

Achievable

This S.M.A.R.T. goal characteristic must be addressed and cannot be ignored. It is not enough to "want" to achieve a specific goal; it must actually be within the realm of possibility. The goal may actually be considered a stretch, but still achievable. Ignoring this characteristic sets people up for failure. In our example, the organization considered the amount of changes and effort necessary to improve quality for product ABC. The organization has determined that a more realistic approach for them that would allow them to achieve the end result would be to establish achievable goals for a three year period with a much more realistic upper percentage of improvement set at 5%. This means that the organization is acceptable of a customer satisfaction index of 79 for product ABC. While that may be considered low, to this organization it is acceptable, especially since they plan on removing product ABC from the market sometime over the next 8-10 years. They will still be able to generate an acceptable revenue stream while R&D is creating a much more robust replacement. Restating the sub-goals for the customer satisfaction index means they might look something like the following:

  • Improve customer satisfaction for year 1 from 75 to 76 resulting in a 1% improvement.
  • Improve customer satisfaction for year 2 from 76 to 77.5 resulting in an overall 3% improvement.
  • Improve customer satisfaction for year 3 from 77.5 to 79.1 resulting in an overall 5% improvement.
  • Maintain a customer satisfaction index of 79 or higher each year until product ABC is removed from the market.

Do not forget that once the business goal is achieved, it must be maintained. In order to maintain it, it must be monitored to prove that it is being maintained.

Relevant

This characteristic focuses on the relevancy of the goal. The specified sub-goal must be relevant to the broader context of the goal. Using an example from above, a goal that would not be relevant might look like, "Improve team efficiency by 5%." While that might be a worthwhile endeavor, in this organization it is not relevant to improving customer satisfaction. The dissatisfaction is based on product quality and focusing on team efficiency may actually prove detrimental to improving customer satisfaction. Address the most critical need, and then focus on the other needs of the organization.

Time-bound

Regardless of the many typical management mandates issued in most organizations, change takes time. It is not as easy as reading a book or taking a class to make the transition complete. In IT organizations, change in behavior is deep seated and will result in varying levels of resistance. You are attempting to move people from their sphere of comfort into an unknown environment.

Therefore, the time given to a change initiative must take into account the effect of change and managing the necessary changes to bring about the successful accomplishment of the goals. However, there must be a time limit established. The amount of time given to the effort must be commensurate with the amount and effect of the change.

For this example, customer satisfaction index is calculated once every year in February. And the current month is October. There are only 3 months to get the changes instituted and the product fielded with a sufficient number of current and future customers to have an adequate amount of time to react to the improved product. Being real means it won't happen. That means the organization has 15 months to get to the first derived measure, a score of 76 for customer satisfaction.

Measures

Using S.M.A.R.T. attributes and the GQM technique allows organizations to develop measurable sub-goals and identify the data needed for the measures used to monitor progress towards the sub-goals. From that, any necessary calculations needing performed along with how this will be reported to management, are defined and all are unique to an organization.

Once business goals are identified at the appropriate level for measuring, operational groups within IT decompose these measurable goals into a set of goals for their own group using S.M.A.R.T. and GQM approaches. These operational goals are at a lower abstraction level than the business goal they trace to and are specific to the organizational contributions necessary to meet the business goal. The operational groups then identify the measures to be used for monitoring progress towards the achievement of their own goals.

After the operational goals and measures are identified, processes are identified along with objective and subjective measures for monitoring them. At the individual and team level, one or more development practices are identified with associated measures for monitoring progress. These are explicitly traced to the appropriate operational goals. This completes the explicit tracing between business goals, operational goals, and practices.

Using S.M.A.R.T. and GQM techniques at each level of a multi-tiered performance measurement system provides the framework of proof to senior management that changes at individual and team levels explicitly impact a specific business goal.