DIY Process Assessment Execution – Analyzing Results and Evaluating Maturity Levels

In the previous post, I gave an example of process assessment survey. Using a one-to-five scale model, you can arrive at a weighted (or a simple average) score for a given process after collecting the data from the assessment participants. The more data points (or survey results) you can collect, the more realistic (and hopefully accurate) the process maturity score will be. Before you take the process maturity scores and start making improvement plans, I think there two other factors to consider when analyzing and evaluating the overall effectiveness of your processes. The two additional factors are:

  • Perceived importance of the process:

In addition to measuring the maturity level of a process, I think it is also important to measure how your customer and the business perceive the importance of the processes. The information gained from measuring the perceived importance can be important when gauging the level of and prioritizing the investments that should go into the improvement plan for a process. For example, a process with a low maturity level but perceived to be of high importance to business may be a good candidate for some serious and well-planned investment. On the other hand, a process, that has a high maturity level in IT’s eyes but perceived to have a lower importance to business, may signal you to have a further look at the current investment level and see whether some scaling-back or reallocation of the funds could be an option. After all, we want to be in a position where the investments of any process will yield the most value for the organization overall. We simply cannot make decisions on the improvement plans without understanding the perceived business values.

Measuring the perceived importance accurately requires asking the right questions and getting the feedback from the right audience. People from the senior management team or IT customers who are considered power users are probably in a better position than others to provide this necessary insight. Also, simply asking IT customers how important a process is to the organization may not be effective because those customers are not likely to be as familiar with the nitty-gritty IT processes as we are. We will need to find a way to extract the information by stating the questions in a way that our customers can understand and respond, without getting into too much of technical jargons.

As an example, the result of this analysis could be a bar chart showing the maturity level and the perceived importance level for the processes under assessment.

  • Degree of Integration Between Processes

Another factor to consider before taking a process maturity score and making an improvement plan is to also understand how well processes integrate with one another. Most ITSM processes rarely act alone, and the effectiveness of an overall ITSM program also depends on the level of integration between processes. Assessing how well one process integrates with another generally involves looking at just how well the output from one process is used in other processes. Some examples of process integration for problem management can include:

    • Processes Providing Input Into Problem Management:
      • Capacity management process could provide historical usage and capacity trends information to aid the root cause analysis or formulation of permanent solutions.
      • Incident management process could provide incident details for the root cause analysis activities. Incident data can also enable proactive problem management through the use of trend analysis.
      • Configuration management process could provide relationship information between configuration items, which can help in determining the impact of problems and potential resolutions.
    • Processes Receiving Output from Problem Management:
      • Incident management process could receive known error records and details of temporary fixes in order to minimize the impact of incidents.
      • Change management process could receive requests for change triggered by problem management to implement permanent solutions to known errors.

What scale should you use to rate the integration between processes? I think a simple scale of one to five should work just fine. For example:

    • One could indicate the output from the originating process is used inconsistently by the target process
    • Two could indicate the output from the originating process is used consistently but only informally by the target process
    • Three could indicate the output from the originating process is both consistently and equally by the target process in a documented manner
    • Four could indicate the output from the originating process is used consistently to support the target process in a managed way
    • Five could indicate the output from the originating process is used consistently to support the target process in an optimized way

You define what the scale really means for your environment in a way that is easily understandable by your team. Also keep in mind that not all processes must integrate seamlessly with other processes on every possible front in order to have an effective ITSM program; however, a good use of the integration scores can help us to uncover opportunities to capitalize on our strengths or to improve upon our challenges. For example, a low integration score between incident and problem management processes could signal us an opportunity to improve how those two processes exchange and consume the output from one another. If we find the known error database is not being utilized as much as we should during incident triage, we should dig in further and see what actions we can take to improve the information flow. If the problem management process is being hampered due to lack of accurate incident information coming from the incident management process, the integration score should point us to the need that we need to raise the quality of information exchange between the two processes.

As an example, the result of the process integration analysis could be a two-by-two chart showing the integration scores between processes.

We have come a long way in this DIY process assessment journey, from gathering the potential resources, planning for the assessment, executing the assessment, to analyzing the results. For the next and concluding post on the process assessment topic, we will discuss presenting the assessment results and suggesting some quick-win items to consider as part of the follow-up activities.

Leave a Reply