What Makes a KPI Special

A key performance indicator (KPI) is a vital tool for an organization seeking grow smart profits, expand customer loyalty, and build a scalable workforce.  While many companies stick a chronological set of numbers on a chart to project it on a wall in front of stakeholders, the reality is what they are seeing may not be a true KPI.   Its just a number unless its special.

To be special, the number needs to have most of the following characteristics in common:

  1. Represents a hierarchy either by time, or by dimension
    1.  Example: Sales broken by Year, Quarter, Month, Week or even day, then sliced by Sales Region
  2. Directly actionable with each stakeholder holding a piece of the action.
    1. Example: Sales are down 15% from goal, Sales Managers, Marketing Coordinators, etc can all speak to how their actions influenced the number
  3. Has  common definition across departments throughout the company
    1. Example: A sale is an action by a user from a variety of channels and promotions that resulted in payment and excludes affiliates for instance.  This definition is signed off by Marketing, Customer Service, Sales, and Product.
  4. Does not include hidden meaning and/or does not hide a trend
    1. Example: A number compared year over year, month over month completely ignores a trend that can be alarming.  While sales could be up year over year, this metric could hide a sudden drop in sales from the beginning of the year.
  5. When displayed with other metrics on a dashboard, the number represents a part of the story and does not represent a conflict to other metrics.
    1. Example: A number is where a chart shows Sales sky-rocketing, but another chart shows New Revenue way down, and yet another charts show Average Deal Size (ADS) flat.  The three charts give conflicting information, so one or more can’t be a true KPI for this business.  In fact, in this case, Sales is the corrupted KPI as it does not conform to a standardized definition.
  6. Supporting data is transparent
    1. Example: Analysts should be able to review aggregate data that rolls up to the KPI for audit reasons.  It provides transparency and allows for drill down capabilities. Often the Analysts receive direct support from a data team are sourced from data sources throughout the business.
  7. The number evolves with the business
    1. Example: A number is just a number unless it can change readily with the business. A team supports the KPI, new data is added when created, and definitions evolve as the understanding of the business evolves.
  8. A goal can easily be set and tracked from the number
    1. Example: If the number is fully understood, a goal can be set and tracked against.  If you can’t answer, “Where do we need to be by the end of the Year?”, then its a number, not a KPI.

Throughout my career, I’ve seen numbers and I’ve seen KPI’s.  The most successful companies I worked with not only have a data team to support their KPIs, but they engage in regular discussions of the KPIs at all levels of the business.  Successful, scalable, and profitable businesses are the ones using special numbers at their core.

When was the last time you saw a number masquerading as a KPI?

Data Perspectives – Trial Users

One of the keys to getting customers hooked on your SaaS product is offering a free trial.  Letting someone experience your product for free for seven to thirty days is a great way to establish trust with the potential customer, let them experience the product, and also gain insight into how they will use the product (customer segmentation).

On a recent project, I was reviewing data for a client and noticed a very interesting pattern in the login histories (not really, but we will call it logins since the real data can’t be shared) for trial users.  This particular client offered a 7-day credit card trial with auto convert to a selected plan (i.e. monthly or annual).    What I expected was a nice curve from day 1, declining each day, relatively smoothly and then an increase in logins after conversion.

However, after summarizing the login data for the first ten days of service (including 3 days for the auto convert), I found a sharp decrease in logins from day 1 and day 2, as well as a blip on day 6. See the chart below.

What was even more fascinating is how the other analysts and “experts” at the company interpreted this data.  Some of the comments are below:

  • “Wow, people pay us and use less?” – referring to the drop is usage on day 8 after becoming a paying customer
  • “Those auto convert reminder emails are working, driving usage!” – referring to the increase in logins one day prior to trial end on day 6
  • “Looks like we need pay per login” – referring to the sharp decline in logins from day 1 to day 2
  • “If we can get the customer to use beyond day 4, we have them!” – not sure how this really fits in as we haven’t correlated logins with LTV, yet
  • “People are cheap” – referring to the people logging in on day 6 to use the product prior to cancellation
  • “If you are going to login to cancel your auto convert on day 6, wouldn’t you try the product one last time?” – again, referring to day 6

The chart is quite simple, a single line with 10 data points.  What isn’t simple is really what this data means.   In fact, I don’t think we can make a decision directly from this data. Rather we need to further understand what the trial users are actually doing on day 6 and how users with logins on day 7 compare to the users on day 1 (is this a bad a marketing channel).  It would also be great to dive into patterns of logins just prior to churn or trial cancellation.

What fascinates me the most, is not only the different perspectives on the data, but the deeper questions that come out of the data.  Data and customer insights are evolutionary.  The more you know, the more you ask questions and the more the decisions and knowledge evolve.

Visualizing a Weather Forecast: WeatherSpark

Media_httpcfweathersp_xibgf

Came across this awesome new weather forecasting site that really lets the visualization geek get intimate with the weather forecast. WeatherSpark is in beta and uses historical weather to predict the weather going forward, offering not only a map, but charts to boot! Check it out.

Training Dashboard Iteration #1 | TrainingMetrix

I thought I would show off a triathlon training dashboard that incorporates all aspects of performance at a high level.  The back end to this is proprietary, but it illustrates the importance of looking at the larger picture when achieving peak performance is crucial.

Iteration #1 – TrainingMetrix Analytic Performance Dashboard

There are six key pieces of information in the above dashboard for the athlete to digest (from top left, clockwise):

  • Weight and TM Performance score.
  • TM Performance Score
  • Daily Workout Score Plot
  • Upcoming Events
  • TM Performance Variable
  • Weekly Goals

How do all of these pieces of information work together? Well, in the above dashboard, you can see a customized example of a triathlete that is rather inconsistent with their training, their life, and therefore their weight and performance score are trending in the wrong direction. This triathlete needs to focus on:

  • Consistency of workouts, which will help reduce their life and workout scores
  • Eating a consistent healthy diet, reducing their nutrition score

Focusing on these two variables, the triathlete can then start tracking toward improved performance and reduce their TrainingMetrix Performance Score.

Dashboards apply to much more than just business related use cases. Fitness and triathlon training can benefit greatly from a well designed dashboard showing analysis from comprehensive data collection. TrainingMetrix (the company I recently founded) is one of the most comprehensive fitness tracking solutions on the market. I developed it from my three years of triathlon training. For more information, visit http://www.trainingmetrix.com

FlowingData: How to Make Bubble Charts

Media_httpflowingdata_vfghc

FlowingData has a great tutorial on creating bubble charts using R. Bubble charts are like scatter plots, but with a third dimension, size of data point. You are able to tell a greater story using bubble charts as opposed to more traditional charts.

Of course, the latest versions of Excel (PC & Mac) have bubble charts built-in.

Cheers!