Every Figure Must Be Good, Bad or Ugly

You get to hear “actionable insights” whenever analytics or roles of data scientists are discussed. It may reach the level of a buzzword, if it hasn’t gone there already. But what does it mean?

That's one ugly number
That’s one ugly number

You get to hear “actionable insights” whenever analytics or roles of data scientists are discussed. It may reach the level of a buzzword, if it hasn’t gone there already. But what does it mean?

Certainly, stating the obvious doesn’t qualify as insightful reporting. If an analyst is compelled to add a few bullet points at the bottom of some gorgeous chart, it has to be more than “The conversion rate decreased by 13.4 percent compared to the same period last year.” Duh, isn’t that what that plot chart is saying, anyway? Tell me something we can’t readily see.

And the word “actionable” means, “So, fine, numbers look bad. What are we supposed to do about it?” What should be the next action for the marketers? Should we just react to the situation as fast we can, or should we consider the long-term effect of such an action, at this point? Shouldn’t we check if we are deviating from the long-term marketing strategies?

Many organizations consider a knee-jerk reaction to some seemingly negative KPI “analytics-based,” just because they “looked” at some numbers before taking action. But that is not really analytics-based decision-making. Sometimes, the best next step is to identify where we should dig next, in order to get to the bottom of the situation.

Like in any investigation, analysts need to follow the lead like a policeman; where do all of these tidbits of information lead us? To figure that out, we need to label all of the figures in reports — good, bad and ugly. But unlike policework, where catching the bad guy is the goal (as in “Yes, that suspect committed a crime,” in absolute terms), numbers in analytics should be judged in a relative manner. In other words, if the conversion rate of 1.2 percent seems “bad” to you, how so? In comparison to what? Your competitors in a similar industry? Last year’s or last quarter’s performance? Other similar product lines? Answering these questions as an analyst requires full understanding of business goals and challenges, not just analytical skillsets.

Last month, at the end of my article “Stop Blaming Marketing Problems on Software,” I listed nine high-level steps toward insight-driven analytics. Let’s dig a little further into the process.

Qualifying numbers into good, bad and ugly is really the first step toward creating solutions for the right problems. In many ways, it is a challenging job — as we are supposed to embark on an analytical journey with a clear problem statement. During the course of the investigation, however, we often find out that the original problem statement is not sufficient to cover all bases. It is like starting bathroom renovation in a house and encountering serious plumbing problems — while doing the job. In such cases, we would have no choice but to alter the course and fix a new set of problems.

In analytics, that type of course alteration is quite common. That is why analysts must be flexible and should let the numbers speak for themselves. Insisting on the original specification is an attitude of an inflexible data plumber. In fact, constantly “judging” every figure that we face, whether on a report or in the raw data, is one of the most important jobs of an analyst.

And the judgment must be within the business context. Figures that are acceptable in one situation may not be satisfactory in another situation, even within the same division of a company. Proper storytelling is another important aspect of analytics, and no one likes to hear lines out of context — even funny ones.

It may sound counterintuitive, but the best way to immerse oneself into a business context is to figure out why the consumer of information is asking certain questions and find ways to make her look good in front of her boss, in the end. Before numbers, figures, fancy graphics, statistical methodologies, there are business goals. And that is the key to determining the baselines for comparisons.

To list a few examples of typical baselines:

  • Industry norm
  • Competitors
  • Overall company norm
  • Other brands
  • Other products/product lines
  • Other marketing channels (if channel-driven)
  • Other regions and countries (if regional)
  • Previous years, seasons, quarters, months, weeks or year-to-date
  • Cost factors (for Return on Investment)

Then, involved parties should get into a healthy argument about key measurements, as different ones may paint a totally different picture. Overall sales figure in terms dollars may have gone down, but the number of high-value deals may have gone up, revealing multiple challenges down the line. Analysts must create an environment where multi-dimensional pictures of the situation may emerge naturally.

Some of the obvious and not-so-obvious metrics are:

  • Counts of opens, clicks, visits, pages views, shopping baskets, abandonments, etc. Typical digital metrics.
  • Number of conversions/transactions (in my opinion, the ultimate prize)
  • Units sold
  • # Unique visitors and/or customers (very important in the age of multichannel marketing)
  • Dollars — Total paid, discount/coupon amount, returns (If we are to figure out what type of offers are effective or harmful, follow the discounts, too.)
  • Days between transactions
  • Recency of transactions
  • Tenure of customers
  • Cost

If we conduct proper comparisons against proper baseline numbers, these raw figures may reveal interesting stories on their own (as in, “which ones are good and which ones are really ugly?”).

If we play with them a little more, more interesting stories will spring up. Simply, start dividing them with one another, again, considering what the users of information would care about the most. For instance:

  • Conversion rates — Compared to opens, visits, unique visitors (or customers), mailing counts, total contact counts, etc. Do them all while at it, starting with the Number of Customers, divided by the Number of Total Contacts.
  • Average dollar per transaction
  • Average dollar per customer
  • Dollar generated per 1,000 contacts
  • Discount ratio (Discount amount / Total dollar generated)
  • Average units per transaction
  • Revenue over Cost (good, old ROI)

Why go crazy here? Because, very often, one or two types of ratios don’t paint the whole picture. There are many instances where conversion rate and value of the transaction move in opposite directions (i.e., high conversion rate, but not many dollars generated per transaction). That is why we would even have “Dollar generated per every 1,000 contacts,” investigating yet another angle.

Then, analysts must check if these figures are moving in different directions for different segments. Look at these figures and ratios by:

  • Brand
  • Division/Country/Territory/Region
  • Store/Branch
  • Channel — separately for outbound (what marketers used) and inbound (what customers used)
  • Product line/Product category
  • Time Periods — Year, month, month regardless of the year, date, day of the week, daypart, etc.

Now here is the kicker. Remember how we started the journey with the idea of baseline comparisons? Start creating index values against them. If you want to compare some figures at a certain level (say, store level) against a company’s overall performance, create a set of index values by dividing corresponding sets of numbers (numbers in question, divided by those of the baseline).

When doing that, even consider psychological factors, and make sure that “good” numbers are represented with higher index values (by playing with the denominators). No one likes double negatives, and many people will have a hard time understanding that lower numbers are supposed to be better (unless the reader is a golfer).

Now the analyst is ready to mark these figures good, bad and ugly — using various index values. If you are compelled to show multiple degrees of goodness or badness, by any means, go right ahead and use five-color scales.

Only then, analysts should pick the most compelling stories out of all of this and put them in less than five bullet points for decision-makers. Anything goes, for as long as the points do matter for the business goals. We have to let the numbers speak for themselves and guide us to the logical path.

Analysts should not shy away from some ugly stories, as those are the best kind. If we do not diagnose the situation properly, all subsequent business and marketing efforts will be futile.

Besides, consumers of information are tired of the same old reports, and that is why everyone is demanding number geeks produce “actionable insights” out of mounds of data. Data professionals must answer that call by making the decision-making process simpler for non-analytical types. And that endeavor starts with labeling every figure good, bad or ugly.

Don’t worry; numbers won’t mind at all.

Author: Stephen H. Yu

Stephen H. Yu is a world-class database marketer. He has a proven track record in comprehensive strategic planning and tactical execution, effectively bridging the gap between the marketing and technology world with a balanced view obtained from more than 30 years of experience in best practices of database marketing. Currently, Yu is president and chief consultant at Willow Data Strategy. Previously, he was the head of analytics and insights at eClerx, and VP, Data Strategy & Analytics at Infogroup. Prior to that, Yu was the founding CTO of I-Behavior Inc., which pioneered the use of SKU-level behavioral data. “As a long-time data player with plenty of battle experiences, I would like to share my thoughts and knowledge that I obtained from being a bridge person between the marketing world and the technology world. In the end, data and analytics are just tools for decision-makers; let’s think about what we should be (or shouldn’t be) doing with them first. And the tools must be wielded properly to meet the goals, so let me share some useful tricks in database design, data refinement process and analytics.” Reach him at stephen.yu@willowdatastrategy.com.

2 thoughts on “Every Figure Must Be Good, Bad or Ugly”

  1. This is probably the best of Stephen’s articles which is a very high bar indeed. It should be required reading for everyone who comes in contact with data and analytics.

    1. Thank you, Peter! I wrote this piece for aspiring analysts and data scientists, as they need to develop both sides of their brains in the age of data.

Leave a Reply

Your email address will not be published. Required fields are marked *