Public:Phanousk/Documentation/RT/metrics

This page is a loose translation of LPS:RT Metriky, which is our primary documentation, and maybe is not up-to-date. Corresponding script (with czech comments) is here: script. Different ZCU's RT queues are treated in different fastness and quality, in fact according to attitude of their operators. If we want to know how each queue stands on the basis of requestors needs than it is likely to analyze all the requests. We made some simple description of metrics quantifying process of treating requests. However it is likely to read the acquired numbers correctly because every queue has unique character and what is considered bad in one queue doesn't have to be bad in other.

Basic metrics

Overview of defined metrics and the method of it's calculation.

• Used variables
  • $stat_period - time for metric's calculation
  • $decay_period - time after that is ticket treated as rotten (not being solved)
  • $multi_requestor - minimal number of requestors to treat a request as a multiuser request
  • @internal_emails - the list of e-mails considered as internal (for IT centre - @civ.zcu.cz)
  • @email_address - the list of requestors e-mail adresses

Evaluation of actual queue status

• M1 - number of unresolved requests
  • (ticket.status == open) || (ticket.status == new) to the date of metric's calculation
  • It express the number of request that are not resolved yet - the high number can be considered as the requests are not followed through or as the requests are only not to set as resolved.
  • The less the better
• M2 - The number of tickets not being solved
  • (ticket.status == new) to the date of metric's calculation
  • The number of tickets which are untouched - the high number means that nobody is working on tickets
  • The less the better
• M3 - The number of tickets without reply
  • (ticket.status == open) || (ticket.status == new) by the date of metric's calculation there is no transaction of "Correspond" type which transaction.EmailAddress is from the group @internal_emails
  • Includes requests where requestor has no reply from IT centre (so he is not informed). Partially overlaps with M1.
  • The less the better
  • Note: M3-M2 are openet tickets without reply, eg typically includes a comment but no correspondence. So somebody might read or solve the case but didn't bother with informing requestor.
• M4 - The number of unresolved tickets from outside
  • Same as M1 but on top of that at least one requestor is not from @internal_emails group.
  • Ticket not being resolved which are requested by people outside the IT centre. High number can indicate users dissapointment.
  • The less the better
• M5 - The number of unresolved tickets from inside
  • Same as M1 but on top of that all requestors are from @internal_emails group.
  • Unresolved tickets being requested by different IT centre departmets or members. High number indicates the scale of bad assistance of IT staff members and therefore the bad mood in the department.
  • The less the better
• M8 - The number of unresolved tickets without owner
  • ((ticket.status == open) || (ticket.status == new)) && (ticket.owner == "") to the date of metric's calculation.
  • The number of tickets which are not taken by anybody. If there is a rule for a queue and it's solvers to take tickets, the high number witnesses about breaking the rules or insubordinance.
  • The less the better but it depends on queue rules
• M9 - The number of unresolved tickets with the last message from requestor
  • (ticket.status == open) || (ticket.status == new) to the date of metric's calculation ad the last transaction.EmailAddress is from some requestor
  • The last correspondence from requestor means that requestor probably needs some reaction from the IT centre. Too big number is suspicious, too many actions are on the side by IT centre. Could mean for example bad management of beckups persons if someone is on the vacation.
  • The less the better
• M10 - The number of tickets not being solved longer than required interval
  • (ticket.status == open) || (ticket.status == new) to the date of metric's calculation and the last requestor's transaction @email_address is not in interval <now-$decay_period, now>
  • The requests which are not touched by IT centre more time than is the required interval. The number means the "rotten" requests, nobody bothered with solving them or at least with commenting them.
  • The less the better
• M15 - The number of unresolved tickets with more requestors
  • (ticket.status == open) || (ticket.status == new) and the number of requestors is at least $multi_requestor
  • The number of essential (large scaled) requests. It can sign the small attention for critical requests.
  • The less the better
• M6 - The time of waiting of unresolved tickets
  • The time from the last queue solver's activity. If he didn't react at all then from the last activity of requestor.
  • It gives an overview about the time user is waiting for solution.
  • The less the better
  • M16b - Minimal interval of reply
  • M16c - Maximal interval of reply
  • M16d - Average interval of reply

Evaluation of the past period

Volume of requests

• M11 - The amount of requests created in a given period
  • The "Create" transaction is in time interval of <now-$stat_period, now>
  • It gives the some overview of request flow in queue
• M12 - The amount of requests resolved in a given period
  • The last "Resolve/Delete/Stall/Reject" transaction is in time interval of <now-$stat_period, now>
  • It gives the some overview of request flow in queue
• M13 - The growth of requests amount in a given period
  • M11-M12
  • The positive number means that more requests were created than resolved, eg the queue grows. Long lasting grow of queue can be the sign of low working morale or about overloading of workers.
  • The less the better
• M14 - The rate of resolved tickets in a given period
  • M12/M11 (for M11 > 0)
  • Analogous rating as M13 bud it is relative (under 100% means growing of queue)
  • The more the better

Time metrics

• M16 - The characteristics of replies in a given period
  • Note: if the sequence of events is like requestor.correspond1, requestor.correspond2, it_centre.correspond1, then is the waiting time calculated as it_centre.correspond1 - requestor.correspond1;. If the requestor is not the same as ticket creator (typically the BSA reports which creates a ticket for someone else), it cannot be paired and those tickets are skipped.
  • M16a - The amount of all replies in a given period
  • M16b - Minimal time of reply
  • M16c - Maximal time of reply
  • M16d - Average time of reply
• M17 - The characteristics of the first replies in a given period
  • M17a - The amount of the first replies in a given period
    • The number of tickets which first "Correspond" transaction from user from @internal_emails is in the time interval of <now-$stat_period, now>. It is according to all tickets, even the resolved ones.
    • It provides some overview about metrics M17b, M17c, M17d
  • M17b - Minimal time needed for the first reply
    • The minimal time difference between request creation and first "Correspond" transaction by the user from @internal_emails, while "Correspond" is in the time interval of <now-$stat_period, now>. It is according to all tickets, even the resolved ones.
    • Minimal period which IT centre needed for the first contact of requestor. It is some overview of the queue with most active solvers or with the most simple tasks :).
    • The less the better
  • M17c - Maximal time needed for the first reply
    • The maximal time difference between request creation and first "Correspond" transaction by the user from @internal_emails, while "Correspond" is in the time interval of <now-$stat_period, now>. It is according to all tickets, even the resolved ones.
    • Minimal period which IT centre needed for the first contact of requestor. Theoretically it shouldn't be longer than maximum of free days in the given period. Too high value means the problem with replying to requestor.
    • The less the better
  • M17d - Average time needed for the first reply
    • Similar to M6 but the average is better for evaluating the attitude to solving tickets in queue.
    • The less the better
• TODO
  • M7 loose subjects
  • number of ticket which's creator is not one of requestors

Metrics calculation

The procedure of acquiring of metrics

TODO

Time evaluation of metrics progress

It is more useful to drill some information from metrics for some period instead of only onetime data.

TODO