Photo by Douglas Bagg / Unsplash

Questions about Metrics

Questions Workshop Apr 20, 2022 (Apr 21, 2022) Loading...

Background materials to the Questions Workshop

Basics

  • What is this a proxy for?
  • Why is this a good proxy?
  • What does this  measurement tell us?
  • What situations suit this metric?
  • Does this metric lead you to enquiries, or offer answers?

Alerts

  • Under what circumstances might it stop being a good proxy?
  • What does this metric leave out?
  • Under what circumstances might it decieve you?
  • How might someone else use it to to game the system, or to deceive you?

Practicalities

  • How do you measure this metric?
  • Does it need to be measured regularly to make sense?
  • How frequently should the measure be taken?
  • Is it noisy, and how do we deal with the noise?
  • Does it aggregate several other metrics?
  • Is it delayed, and what it the lag?
  • Does the metric very by who is measuring it?
  • is this a relative metric, or absolute?
  • What are the units?

Interpretation

  • What does “going up” mean for this metric?
  • What does “going down” mean for this metric?
  • Can it go up and down?
  • What does 0 mean?
  • Can it go negative? What does that mean?
  • Do non-integer values make sense? What do they mean?
  • Is everything it counts as a unit the same size?
  • Is there a minimum? What does minimum mean?
  • Is there a maximum? What does maximum mean?
  • What does no change mean?
  • What does slow change mean? What is slow for this measure?
  • What does fast change mean? What is fast, for this measure?
  • What might it mean if  doing work, over a given period, makes no change to the metric?
  • Can the measure change little, but what is measured change a lot (i.e. there are still 10 failures, but they're all new)
  • What trends are interesting? Over what periods?
  • Is the change, or rate of change, more interesteing than the actual value?
  • What other measures generally change in the same way, or the opposite way? What might be happening in circumstances where this corrleation breaks down?
  • Do you have a model which attempts to explain this metric? WHere does that model break down?

Member reactions

Reactions are loading...

Sign in to leave reactions on posts

Tags

Comments

Sign in or become a Workroom Productions member to read and leave comments.

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.