MENU
complex

Key Metrics for Application Outsourcing – What to Measure!

May 24, 2014 • Key Metrics Application Outsourcing - What to Measure!

Delivery_Scorecard_C

Application outsourcing engagements are typically governed via specific service agreements that are comprised of a number of SLAs. Metrics are a critical ingredient of an SLA. There are dozens of metrics that could potentially be measured for outsourced application support and maintenance services.

 

Tackle the Basics First

 

Days & Hours

Unit of Time – What is a “day”? Does it start in a specific time-zone? Does it include holidays? Does it include weekends?

Define a “day” starting from a specific time-zone such as the time-zone from which most of the application users work. Often, different SLA’s are defined based on different business calendars. For example, you may want 24×7 support for Severity-1 production support, whereas regular business hours may be adequate for requests that are not Severity-1.

Single Source of Truth

Choosing the right metrics is important. Equally important is determining upfront how these metrics will be tracked and reported.

Allocate time and budget to create an application services governance portal that enables periodic update of metrics, and reporting of metrics to different stakeholders.

Should I Use My Provider’s Portal?

Where you have a limited scope outsourcing to a single provider, using the reporting tools provided by the provider is fine. When working with multiple providers, or when a combination of internal teams and outsourced teams are used to deliver services, you are better off with a dedicated tool owned by your organization.

How, How Often & When

For every metric you select, you need to determine the source of the metric, how often the metric should be collected, when it should be collected, and whether it is an input metric, a process metric or an output metric.

Setup an application services governance portal that has a services catalog with metrics selected for each service.

How Do You Verify the Measurement?

For every metric you select, you also need to determine how you can verify that the metric has been correctly collected. Do you trust the source of the data? Can you audit it?

Ensure your SLA allows you to periodically audit the source of the metrics. Leverage the services portal to schedule and track these audit results.

How Much Control?

Managers who are used to managing delivery will find the transition to governance challenging. For instance, there may be high attrition in your outsourced delivery team – should you care? Should you have final say on who is part of your delivery team and who is not?

Govern through results – unless the results (or trends) are indicators you need to take more control. Retain the right to do so contractually.

 

Operational Support Services – Production Support, User Support

Business Demand Metrics

These are input metrics that indicate how much work is requested.

User Support Work Backlog

This is the total volume of user support requests (submitted in a specified time period) that remain open and unassigned.

If the user support backlog increases between multiple time-periods, it is an indicator that the application may not be designed or documented well, or may have significant gaps between actual and desired functionality. It may also be an indicator of inadequate support staffing, or a growing user community.

Production Support Work Backlog

This is the total volume of production support requests (submitted in a specified time period) that remain open and unassigned.

If the production support backlog increases between multiple time-periods, it is an indicator that the application may not operate well, or there may be gaps in understanding of actual versus projected capacities.

Performance Metrics

These are output metrics that indicate how much work is done. A high volume of requests closed in a time period may not be a reason to rejoice; it may point to other problems that are leading to a high demand for this service.

User Support Work Volume

This is the total volume of user support requests closed in a specified time period.

This metric is an indicator of how much work the user support team is completing in a time period.

Production Support Work Volume

This is the total volume of production support requests closed in a specified time period.

This metric is an indicator of how much work the production support team is completing in a time period.

Quality Metrics

These are output metrics that help you determine how well the work is performed.

Re-Work Requests

This is the total volume of production support requests that are re-opened in a specified time period.

The request tracking system may not allow requests to be re-opened; see if you can link new requests to existing requests that have been re-opened.

Efficiency Metrics

These metrics tell you how efficient your delivery is.

Cost Work Efficiency

This is a measure of how well your delivery is performing.

The Total Amount of Work can be measured as requests or to be more accurate, should be measured in actual work hours.

Cost Work-Efficiency = Total Cost of Work / Total Amount of Work

First-Time Right

This is a measure of how trained your delivery team is, and how good is your knowledge-base. It may also point to better end-user training needs.

A metric value sustained below 80% over a period of time should trigger an evaluation of the skills and training of your delivery staff.

Utilization Metrics

These metrics tell you how effective your delivery is.

Total Work Effectiveness

This metric tells you whether you have excess capacity or are stretched thin.

Sustained utilization that exceeds 95% or is lower than 70% is a warning signal. Also, be aware that your provider may provide data that indicates higher utilization than what is actually the case; see if you can map actual work performed to the time reported.

Total Work Effectiveness = Total Amount of Work / Total Amount of Capacity

Customer Satisfaction Metrics

These are output metrics that help you determine the perception of the recipients of the service.

Customer Satisfaction Index

This metric should be derived from a weighted set of responses that evaluate the performance of the provider across a range of performance and relationship factors.

Perception can often become reality; often in production support, what will be remembered is the time the business application was down, not when it was up. Providing real-time visibility into service demand, service delivery and key performance indicators is a good way to keep reminding and communicating with business users the value of the service delivered.

Business Impact Metrics

These are key output metrics that indicate to you how well the application is performing for the business. It is often difficult to compute these metrics – and most organizations focus on operational metrics rather than business impact metrics. Though, even with basic assumptions, it is recommended you track these metrics.

Application Downtime Impact

This is a key business impact metric as the key function of production support is to ensure availability of applications to the business.

Application Downtime Impact = (Average Value per Business Transaction * Average Number of Business Transactions per Hour during that period * Total Downtime Hours)

For example, if a mortgage processing company has average revenue of $5000 per loan processed, and it processes on the average 100 loans per business hour, the application downtime impact of one business hour is:

Application Downtime Impact = $5000 * 100 * 1 = $500000.

Application Response Impact

The application may be available, though far too slow for typical business usage. It is harder to compute this metric, and attribute a cost to the sluggishness of the application.

Application Response Impact = (Expected Number of Business Transactions – Actual Number of Business Transactions) for a time period * Average Value per Business Transaction in the same time period.

Total Business Impact

This is the combined cost impact of application downtime, and poor application response.

Total Business Impact = Application Downtime Impact + Application Response Impact

Application Maintenance Services – Enhancements and Maintenance

Application maintenance services enable customers to adapt the applications in accordance with changing or expanding business needs. In an ERP rollout – an enhancement may mean adding a region or country to the ERP application. A maintenance request is a defect in the application.

Customer Demand Metrics

These are input metrics that indicate how much work is requested.

Enhancements Work Backlog

This is the total volume of enhancement requests (submitted in a specified time period) that remain open and unassigned.

If the enhancement work backlog increases between multiple time-periods, it is an indicator that the application may have significant gaps between actual and desired functionality. It may also be an indicator of inadequate staffing, or poor productivity.

Maintenance Work Backlog

This is the total volume of maintenance work requests (submitted in a specified time period) that remain open and unassigned.

If the maintenance work backlog increases between multiple time-periods, there may be gaps in understanding of actual versus projected capacities.

Priority Request Aging

This is the total volume of priority maintenance work requests (submitted in a specified time period) that remain open and unassigned grouped by number of days since open.

While the backlog may be increasing, business users become dissatisfied when the high priority requests in the backlog are increasing over time. It may also signal inadequate or incorrect prioritization processes.

Performance Metrics

These are output metrics that indicate how much work is done.

Maintenance Work Volume

This is the total volume of maintenance requests (or defects) closed in a time period.

This metric is an indicator of how much work the user support team is completing in a time period.

Enhancement Work Volume

This is the total volume of enhancement requests closed in a time period.

This metric is an indicator of how much new enhancement work the production support team is completing in a time period.

Quality Metrics

These are output metrics that help you determine how well the work is performed. This is not quality of the application, but quality of the service delivered.

Re-Work Requests

This is the total volume of maintenance requests that are re-opened in a time period. The request tracking system may not allow requests to be re-opened; see if you can link new requests to existing requests that have been re-opened.

Budget Accuracy

This is variance of the estimates and the actual work effort.

Whether this difference is computed in monetary cost or time cost, this is an important indicator of how accurately the service team is able to respond to business needs.

Efficiency Metrics

These metrics tell you how efficient your delivery is.

Cost Work Efficiency

This is a measure of how well your delivery is performing.

The Total Amount of Work can be measured as requests or to be more accurate, should be measured in actual work hours.

Cost Work-Efficiency = Total Cost of Work / Total Amount of Work

Utilization Metrics

These metrics tell you how effective your delivery is.

Total Work Effectiveness

This metric tells you whether you have excess capacity or are stretched thin.

Utilization that exceeds 90% or is lower than 70% is a warning signal. Also, be aware that your provider may provide data that indicates higher utilization than what is actually the case.

Total Work Effectiveness = Total Amount of Work / Total Amount of Capacity

Related Posts

21 Responses to Key Metrics for Application Outsourcing – What to Measure!

  1. [...] This post was mentioned on Twitter by Runar Larsson, Outsrce Website and Outsrce Graphics, Bob Anderson. Bob Anderson said: New Blog Post: Key Metrics – Application Outsourcing – What to Measure! http://bit.ly/4J57OY [...]

  2. Social comments and analytics for this post…

    This post was mentioned on Twitter by rbobanderson: New Blog Post: Key Metrics – Application Outsourcing – What to Measure! http://bit.ly/4J57OY

  3. [...] rest is here: ITIL Service Management & IT Application Support | Key Metrics for … Share and [...]

  4. [...] the original: ITIL Service Management & IT Application Support | Key Metrics for … tags: application, are-typically, batch-script, code-module, data, governed-via, help-desk, [...]

  5. [...] post: ITIL Service Management & IT Application Support | Key Metrics for … tags: application, are-typically, batch-script, change-affected, help-desk, incident, [...]

  6. IT Services says:

    That’s interesting that if the production support backlog increases between multiple time-periods, it is an indicator that the application may not operate well. Nice blog.

  7. Bob Anderson says:

    Yes, that can be one indicator. If the support team’s application maintenance / enhancement backlog is continuing to increase. The team lead needs to investigate if just more work is coming in, the team is not getting the backlog consumed as quickly as they should OR is time being spent on fixing Issues / Problems which may have far more serious indications “that the application may not be operating well” – Good job connecting the dots!

  8. Ravindra Joshi says:

    Support requests are raised by customer quite frequenty- in ERP application say 2-3 request per week which was @ 15 request a week when application was rolled out. Can we consider a metrics like no of requests, compared to same time previous year and show the improvment in Suport Servcies?

  9. Bob Anderson says:

    Yes, you could count the number and type of requests by month over the time period since the rollout. Graph the number per month over the time since rolled out. If you want to get more detailed bet a SLA agreement of Time to Respond and Time To Resolve for – Critical, High, Medium, Low requests (different time for each level of importance) Graph the % with in the Service Guidelies and Outside Service Guideliens. You can also calculate a Trend Line of the number received per month this calc is in EXCEL, this is even more impressive to service improvement. Bob

  10. Bob Anderson says:

    Take a look on the Tracer Drop Down Menu for Service Operations. There are a number of Service Goal reports / graphs there that are examples of what we are talking about. Bob

  11. Bob Anderson says:

    Absolutely you can but remember a count of the number of service requests recived is not enough: How large are they, hours, how many people were involved (average FTE), you must compare apples to apples. More data are required beyond just counts.

  12. [...] > Continue reading: Tackle the Basics First and Operational Support Services [...]

  13. Jon says:

    Thanks for any other informative web site. The place else may
    I am getting that kind of information written
    in such an ideal method? I’ve a challenge that I am simply now running on,
    and I’ve been on the look out for such information.

  14. What a material of un-ambiguity and preserveness of valuable familiarity about unexpected feelings.

  15. seo rankings says:

    Can I just say what a comfort to uncover someone that genuinely knows what they’re talking about online.

    You actually understand how to bring a problem to light and make
    it important. A lot more people need to check this out
    and understand this side of your story. I can’t believe you are not more popular because you surely have the gift.

    Here is my website; seo rankings

  16. seo Google says:

    Informative article, exactly what I needed.

    My web blog; seo Google

  17. I have to tell you that it’s hard to find your posts in google,
    i found this one on 17 spot, you should build some quality backlinks in order
    to rank your site, i know how to help you, just type in google – Arshumaker
    SEO tips

  18. oxycontin says:

    Hi there, I enjoy reading through your article.
    I like to write a little comment to support you.

    Here is my blog: oxycontin

  19. oxycodone says:

    Wow, marvelous blog structure! How long have you ever been running a blog for?
    you made blogging glance easy. The whole glance of your site iis magnificent, llet alone the congent material!

    Feel free to surf to my web site – oxycodone

  20. oxycontin says:

    My spouse and I stumbled over here coming from a differennt web
    page and thought I might as well check things out.
    I like what I see so i am just following you. Look forward to finding out about your web page for a
    second time.

    My web blog – oxycontin

  21. oxycodone says:

    It is in reality a nice and hellful piece of info. I’m glad that
    you shared thiss useful information with us.

    Please stay us up to date like this. Thank you for sharing.

    Look at my homepage; oxycodone

Add Comment Register



Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *

« »