From the last DIY assessment post, we discussed the data gathering methods and instruments to use for the surveys, workshops, and interviews. No matter what method(s) you end up deploying for your assessment, you will need a list of good/effective/best practices for a process in order to formulate the assessment questions. During the first post of the series, we talked about what reference sources you can use to come up with a list of good practices for a given process. In this post, we will illustrate an example of what the good practices and survey questions might look like for Problem Management.
As you look through the example document, I would like to point out the following:
- Each question in the questionnaire represents a good practice as part of what a mature process would look like. To come up with the list of practices, I leveraged the information from ISO/IEC 20000 Part 2: Guidance on the Application of Service Management Systems. With helpful information sources like ITIL, ISO 20000, COBIT, etc., they provide a great starting point for us DIY’ers and there is no reason to reinvent the wheels for the most part.
- To rank the responses and calculate the maturity level, I plan to use the 5-point scale of CMMI. The maturity levels used by CMMI include 1) Initial, 2) Repeatable, 3) Defined, 4) Managed, and 5) Optimized. However, the maturity levels will not likely be something your survey audience will know very well, so we need to find some other ways for our survey audience to rank their answers. As you can see from the example, I used either the scale of 1) Never, 2) Rarely, 3) Sometimes, 4) Often, 5) Always or 1) Not at All, 2) Minimally, 3) Partially, 4) Mostly, 5) Completely. You don’t have to use both scales – it all depends on how you ask the questions. I could have asked all questions with the scale of 1) Never, 2) Rarely, 3) Sometimes, 4) Often, 5) Always or vice versa. In my example, I chose to mix things up a bit by using both scales just to illustrate the fact that both scales are viable for what we need to do.
- Some questions are better asked with close-end options like Yes or No, instead of using a scale. Those questions tend to deal with whether you have certain required artifacts or deliverables. For example, you either have documented problem management process and procedures, or you don’t.
- As you can see, the scale questions translate nicely when calculating the maturity level. You may calculate the maturity level by using a simple average of all responses from the scale questions, where all questions have an equal weight or preference. Depending on your environment or organizational culture, you may also assign a different weight to each question by emphasizing certain practices over others. For the close-end questions, you will need to think about what the responses of “yes” and “no” mean when you calculate the final maturity level. For example, you may say having an “Yes” for a group of questions gets a score of 3 out 5, where the response of “no” equal to 1. For some questions, you may even say the “yes” response equals to 5.
- This is a simplistic model for assessing and calculating maturity level for a DIY approach. You will need to construct a similar good practice model for each process you plan to assess. Coming up with a list of good practices model to assess against can turn into a significant time investment. However, the majority of effort is upfront and you can re-use the model for subsequent assessments. If you contract out the assessment exercise to a consultant, coming up with the best practice model to evaluate your processes against is normally a deliverables from the consultant. Be sure to spend some time to understand your consultant’s model, and make sure the best practice model is applicable to your organization. It is an important way to ensure the assessment results will be meaningful and easier for everyone to understand.
Please have a look at the example document and let me know what you would do to improve it. On the next post, we will continue the discussion of the assessment execution phase by examining how to analyze the results and evaluate the maturity Levels. We will also discuss how inter-process integration as well as organization and culture could play a part in the maturity level assessment.