Product Metrics
Estimated Time: 30 minutes
🗣 "If you can't measure it, you can't improve it"
We previously discussed crafting MVPS and using the "Build, Measure, Learn" framework to iterate towards a useful product. For iterative development to deliver customer and business value, software teams have to measure the right things, then learn from them. In this lesson, we explore how to determine the right product metrics to measure.
Measuring the right metrics and drawing useful insights can be challenging. Product teams often make two mistakes when it comes to metrics
- They rely on vanity metrics
- They are data rich but insight poor
Metrics mistakes
Vanity metrics
Vanity metrics may be impressive on the surface but do not provide meaningful insights into the performance of the software. Some examples are:
-
Pageviews: pageviews can be misleading because a website may get a lot of visitors, but it doesn't indicate whether users are actually engaging with the content on the site.
-
Downloads: the number of downloads for a mobile app can be a vanity metric if many users download the app but few actually use it
-
Sign-ups: the number of sign-ups for a service can be a vanity metric if many users sign up but few actually become regular users
-
Social media followers: having a large number of social media followers doesn't necessarily translate into meaningful engagement with the product
-
Time spent on site: the amount of time users spend on a website can be a vanity metric since it doesn't necessarily indicate whether users are finding what they need or taking desired actions.
These metrics may provide some indication of reach, but what makes them "vanity metrics" is that they don't necessarily tell us if the product is meeting the goals of the user or business.
Data rich but insight poor
Being data-rich but insight-poor means that an organization has access to large amounts of data, but is not able to extract meaningful insights. In other words, the organization may be collecting data but cannot use it effectively to inform decision-making or drive improvements.
Avoiding mistakes with actionable metrics
The two metrics mistakes above have the same solution:
Measuring what matters means:
- Tracking metrics that are tied to specific business goals or user needs. These "actionable metrics" help teams understand how the software is performing and how it can be improved.
- Tracking metrics consistently and accurately. If the data being collected is inaccurate or incomplete, it may be difficult to draw meaningful conclusions or make informed decisions based on that data. So, good teams ensure they collect data well.
- Analyzing data regularly: Having access to high-quality data is not enough. Organizations will need to have the skills and make the time to analyze that data effectively.
