How Numbers Lead Us Astray So Easily

Frogs, fish, dogs, spiders, hyenas, chimps and others in the animal kingdom all have an innate ability for counting. But we humans are easily fooled by numbers, especially when they’re presented in context. Learning to exploit the power of context can pay off big for marketers, but at the same time, marketers need to be careful not to be fooled themselves.

numbers
Creative Commons license. | Credit: Pixabay by fotoblend

Frogs, fish, dogs, spiders, hyenas, chimps and others in the animal kingdom all have an innate ability for counting. But we humans are easily fooled by numbers, especially when they’re presented in context. Learning to exploit the power of context can pay off big for marketers, but at the same time, marketers need to be careful not to be fooled themselves.

Consider the example of the Economist subscription offer discussed by Dan Ariely in his book “Predictably Irrational.” Ariely duplicated this subscription offer with a group of 100 MBA students:

Chuck McLeester Chart 1
Credit: Chuck McLeester

Which would you have chosen?

Repeating the exercise without the “decoy” offer of the print only subscription yielded the following results:

Chuck McLeester Chart 2
Credit: Chuck McLeester

Which would you have chosen this time? Clearly, context will fool us into perceiving the value of offers differently.

Fish are not so easily fooled.

“Small fish benefit from living in schools, and the more numerous the group, the statistically better a fish’s odds of escaping predation. As a result, many shoaling fish are excellent appraisers of relative head counts. Three-spined sticklebacks are … able to tell six fellow fish from seven, or 18 from 21 — a comparative power that many birds, mammals and even humans might find hard to beat.” Beastly Arithmetic, NYTimes Feb 6 2018

Psychology-based marketing expert Jeanette McMurtry says,

“When marketers discover the inconsistencies and irrationalities about how consumers make choices, they can create messaging that engages consumers’ minds, both conscious and unconscious. When that happens, there’s a lot more to ‘count’ when it comes to sales, revenue, ROI and lifetime value.”

One of the reasons we’re so easily tricked by numbers is our reliance on verbal intuition. In his book, “Thinking Fast and Slow,” Daniel Kahneman provides several illustrations of how our intuition gets in the way of arithmetic when we’re presented with numerical problems in a verbal context. What is your initial response to Kahneman’s word problem?

  • A ball and a bat cost $1.10
  • The bat costs one dollar more than the ball.
  • How much does the ball cost?

You would not be alone if your initial response was 10 cents. But you would be wrong. Because if the ball cost 10 cents and the bat costs one dollar more than the ball then the total cost would be $1.20.

Consider how you might take advantage of people’s intuitive responses when constructing offers, but don’t let your own intuition get in the way of making decisions. Sometimes marketers are fooled by test results because they look for cause and effect in results that could easily have happened randomly. If you’re testing creative variations with samples of 25,000 impressions and your usual clickthrough rates are in the range of 1 percent (which yields a results pool of about 250 clicks), you should know with that sample size and that average response rate, your results can vary by 10 percent. So, statistically there’s a 90 percent chance that you could have gotten 225 clicks or 275 clicks. Yet, if you got both those extremes in an A/B test, it would be easy to conclude that one cell beat the other by a lot.

We are similarly confused by percentages. Psychologists Rochel Gelman of Rutgers University and Jennifer Jacobs Danan of the University of California, Los Angeles, have studied how often reasonably well-educated people miscalculate percentages. We hear that the price of something rose by 50 percent and then fell by 50 percent, and we reflexively, mistakenly conclude, “Oh good, we’re back to where we started.” Beastly Arithmetic, NYTimes Feb 6 2018

Feel free to comment with your answer to this percentage problem, or with any thoughts or experiences you have on using consumers’ proclivity for intuition over rationality to better your marketing efforts.

Every Figure Must Be Good, Bad or Ugly

You get to hear “actionable insights” whenever analytics or roles of data scientists are discussed. It may reach the level of a buzzword, if it hasn’t gone there already. But what does it mean?

That's one ugly number
That’s one ugly number

You get to hear “actionable insights” whenever analytics or roles of data scientists are discussed. It may reach the level of a buzzword, if it hasn’t gone there already. But what does it mean?

Certainly, stating the obvious doesn’t qualify as insightful reporting. If an analyst is compelled to add a few bullet points at the bottom of some gorgeous chart, it has to be more than “The conversion rate decreased by 13.4 percent compared to the same period last year.” Duh, isn’t that what that plot chart is saying, anyway? Tell me something we can’t readily see.

And the word “actionable” means, “So, fine, numbers look bad. What are we supposed to do about it?” What should be the next action for the marketers? Should we just react to the situation as fast we can, or should we consider the long-term effect of such an action, at this point? Shouldn’t we check if we are deviating from the long-term marketing strategies?

Many organizations consider a knee-jerk reaction to some seemingly negative KPI “analytics-based,” just because they “looked” at some numbers before taking action. But that is not really analytics-based decision-making. Sometimes, the best next step is to identify where we should dig next, in order to get to the bottom of the situation.

Like in any investigation, analysts need to follow the lead like a policeman; where do all of these tidbits of information lead us? To figure that out, we need to label all of the figures in reports — good, bad and ugly. But unlike policework, where catching the bad guy is the goal (as in “Yes, that suspect committed a crime,” in absolute terms), numbers in analytics should be judged in a relative manner. In other words, if the conversion rate of 1.2 percent seems “bad” to you, how so? In comparison to what? Your competitors in a similar industry? Last year’s or last quarter’s performance? Other similar product lines? Answering these questions as an analyst requires full understanding of business goals and challenges, not just analytical skillsets.

Last month, at the end of my article “Stop Blaming Marketing Problems on Software,” I listed nine high-level steps toward insight-driven analytics. Let’s dig a little further into the process.

Qualifying numbers into good, bad and ugly is really the first step toward creating solutions for the right problems. In many ways, it is a challenging job — as we are supposed to embark on an analytical journey with a clear problem statement. During the course of the investigation, however, we often find out that the original problem statement is not sufficient to cover all bases. It is like starting bathroom renovation in a house and encountering serious plumbing problems — while doing the job. In such cases, we would have no choice but to alter the course and fix a new set of problems.

In analytics, that type of course alteration is quite common. That is why analysts must be flexible and should let the numbers speak for themselves. Insisting on the original specification is an attitude of an inflexible data plumber. In fact, constantly “judging” every figure that we face, whether on a report or in the raw data, is one of the most important jobs of an analyst.

And the judgment must be within the business context. Figures that are acceptable in one situation may not be satisfactory in another situation, even within the same division of a company. Proper storytelling is another important aspect of analytics, and no one likes to hear lines out of context — even funny ones.

It may sound counterintuitive, but the best way to immerse oneself into a business context is to figure out why the consumer of information is asking certain questions and find ways to make her look good in front of her boss, in the end. Before numbers, figures, fancy graphics, statistical methodologies, there are business goals. And that is the key to determining the baselines for comparisons.

To list a few examples of typical baselines:

  • Industry norm
  • Competitors
  • Overall company norm
  • Other brands
  • Other products/product lines
  • Other marketing channels (if channel-driven)
  • Other regions and countries (if regional)
  • Previous years, seasons, quarters, months, weeks or year-to-date
  • Cost factors (for Return on Investment)

Then, involved parties should get into a healthy argument about key measurements, as different ones may paint a totally different picture. Overall sales figure in terms dollars may have gone down, but the number of high-value deals may have gone up, revealing multiple challenges down the line. Analysts must create an environment where multi-dimensional pictures of the situation may emerge naturally.

Some of the obvious and not-so-obvious metrics are:

  • Counts of opens, clicks, visits, pages views, shopping baskets, abandonments, etc. Typical digital metrics.
  • Number of conversions/transactions (in my opinion, the ultimate prize)
  • Units sold
  • # Unique visitors and/or customers (very important in the age of multichannel marketing)
  • Dollars — Total paid, discount/coupon amount, returns (If we are to figure out what type of offers are effective or harmful, follow the discounts, too.)
  • Days between transactions
  • Recency of transactions
  • Tenure of customers
  • Cost

If we conduct proper comparisons against proper baseline numbers, these raw figures may reveal interesting stories on their own (as in, “which ones are good and which ones are really ugly?”).

If we play with them a little more, more interesting stories will spring up. Simply, start dividing them with one another, again, considering what the users of information would care about the most. For instance:

  • Conversion rates — Compared to opens, visits, unique visitors (or customers), mailing counts, total contact counts, etc. Do them all while at it, starting with the Number of Customers, divided by the Number of Total Contacts.
  • Average dollar per transaction
  • Average dollar per customer
  • Dollar generated per 1,000 contacts
  • Discount ratio (Discount amount / Total dollar generated)
  • Average units per transaction
  • Revenue over Cost (good, old ROI)

Why go crazy here? Because, very often, one or two types of ratios don’t paint the whole picture. There are many instances where conversion rate and value of the transaction move in opposite directions (i.e., high conversion rate, but not many dollars generated per transaction). That is why we would even have “Dollar generated per every 1,000 contacts,” investigating yet another angle.

Then, analysts must check if these figures are moving in different directions for different segments. Look at these figures and ratios by:

  • Brand
  • Division/Country/Territory/Region
  • Store/Branch
  • Channel — separately for outbound (what marketers used) and inbound (what customers used)
  • Product line/Product category
  • Time Periods — Year, month, month regardless of the year, date, day of the week, daypart, etc.

Now here is the kicker. Remember how we started the journey with the idea of baseline comparisons? Start creating index values against them. If you want to compare some figures at a certain level (say, store level) against a company’s overall performance, create a set of index values by dividing corresponding sets of numbers (numbers in question, divided by those of the baseline).

When doing that, even consider psychological factors, and make sure that “good” numbers are represented with higher index values (by playing with the denominators). No one likes double negatives, and many people will have a hard time understanding that lower numbers are supposed to be better (unless the reader is a golfer).

Now the analyst is ready to mark these figures good, bad and ugly — using various index values. If you are compelled to show multiple degrees of goodness or badness, by any means, go right ahead and use five-color scales.

Only then, analysts should pick the most compelling stories out of all of this and put them in less than five bullet points for decision-makers. Anything goes, for as long as the points do matter for the business goals. We have to let the numbers speak for themselves and guide us to the logical path.

Analysts should not shy away from some ugly stories, as those are the best kind. If we do not diagnose the situation properly, all subsequent business and marketing efforts will be futile.

Besides, consumers of information are tired of the same old reports, and that is why everyone is demanding number geeks produce “actionable insights” out of mounds of data. Data professionals must answer that call by making the decision-making process simpler for non-analytical types. And that endeavor starts with labeling every figure good, bad or ugly.

Don’t worry; numbers won’t mind at all.