Questions about Metrics
Background materials to the Questions Workshop
- What is this a proxy for?
- Why is this a good proxy?
- What does this measurement tell us?
- What situations suit this metric?
- Does this metric lead you to enquiries, or offer answers?
- Under what circumstances might it stop being a good proxy?
- What does this metric leave out?
- Under what circumstances might it decieve you?
- How might someone else use it to to game the system, or to deceive you?
- How do you measure this metric?
- Does it need to be measured regularly to make sense?
- How frequently should the measure be taken?
- Is it noisy, and how do we deal with the noise?
- Does it aggregate several other metrics?
- Is it delayed, and what it the lag?
- Does the metric very by who is measuring it?
- is this a relative metric, or absolute?
- What are the units?
- What does “going up” mean for this metric?
- What does “going down” mean for this metric?
- Can it go up and down?
- What does 0 mean?
- Can it go negative? What does that mean?
- Do non-integer values make sense? What do they mean?
- Is everything it counts as a unit the same size?
- Is there a minimum? What does minimum mean?
- Is there a maximum? What does maximum mean?
- What does no change mean?
- What does slow change mean? What is slow for this measure?
- What does fast change mean? What is fast, for this measure?
- What might it mean if doing work, over a given period, makes no change to the metric?
- Can the measure change little, but what is measured change a lot (i.e. there are still 10 failures, but they're all new)
- What trends are interesting? Over what periods?
- Is the change, or rate of change, more interesteing than the actual value?
- What other measures generally change in the same way, or the opposite way? What might be happening in circumstances where this corrleation breaks down?
- Do you have a model which attempts to explain this metric? WHere does that model break down?
Reactions are loading...
Sign in to leave reactions on posts
Sign in or become a Workroom Productions member to read and leave comments.