Danc, over at Lost Garden, recently made an interesting post describing Improving Bug Triage with User Pain. It describes a bug triaging methodology, and has several good points. I disagree slightly with his release goal for bugs, however.
Here's my comment:
Setting a pain threshold for a release is a very easy policy, but I'm concerned about the build up of low pain bugs. 100 Low pain bugs... ok. Next release, 300 low pain bugs.. and they're starting to have in impact. 1000 low pain bugs, and in aggregate they represent larger pain to me.
An alternative I would suggest is to get the total pain below some value. E.g each bug's pain value is simply it's fractional user pain value. So two bugs at 90% user pain and 20% would total to 1.1 user pain. Set the bar for the release to be something like 50 user pain.
The result is that working on highest user pain bugs reduces the total pain rapidly. But, if there is a huge base of low user pain bugs, it's still recognized and addressed.
Your visualization of a red line at a certain user pain threshold becomes instead a line at the point where the sum of all bugs crosses the total pain threshold.
(image by flicker user Marvin (PA))
I agree with the notion of the total pain level of bugs in a release. You do have to guard against the gradual accumkulation of "low-pain" bugs.
ReplyDeleteIn addition to the notion of "sum-of-the-pain" as a metric, however, I think you need to also look at how many of the bugs are in the same areas. E.g., if you have fifty "small" bugs scatterred across twenty-five areas of the product, that seems like less pain than fifty small bugs in ONE are of the product.