Marketing Metrics Aren’t Baseball Scores

Lester Wunderman is called “the Father of Direct Marketing” — not because he was the first one to put marketing offers in the mail, but because he is the one who started measuring results of direct channel efforts in more methodical ways. His marketing metrics are the predecessors of today’s measurements.

Lester Wunderman is called “the Father of Direct Marketing” — not because he was the first one to put marketing offers in the mail, but because he is the one who started measuring results of direct channel efforts in more methodical ways. His marketing metrics are the predecessors of today’s measurements.

Now, we use terms like 1:1 marketing or digital marketing. But, in essence, data-based marketing is supposed to be looped around with learnings from results of live or test campaigns. In other words, playing with data is an endless series of learning and relearning. Otherwise, why bother with all this data? Just do what you gut tells you to do.

Even in the very beginning of the marketer’s journey, there needs to a step for learning. Maybe not from the results from past campaigns, but something about customer profiles and their behaviors. With that knowledge, smart marketers would target better, by segmenting the universe or building look-alike or affinity models with multiple variables. Then a targeted campaign with the “right” message and offers would follow. Then what? Data players must figure out “what worked” (or what didn’t work). And the data journey continues.

So, this much is clear; if you do not measure your results, you are really not a data player.

But that doesn’t mean that you’re supposed to get lost in an endless series of metrics, either. I sometimes see what is commonly called “Death by KPI” in analytically driven organizations. That is a case where marketers are too busy chasing down a few of their favorite metrics and actually miss the big boat. Analytics is a game of balance, as well. It should not be too granular or tactical all of the time, and not too high in the sky in the name of strategy, either.

For one, in digital marketing, open and clickthrough rates are definitely “must-have” metrics. But those shouldn’t be the most important ones for all, just because all of the digital analytics toolsets prominently feature them. I am not at all disputing the value of those metrics, by the way. I’m just pointing out that they are just directional guidance toward success, where the real success is expressed in dollars, pounds and shillings. Clicks lead to conversions, but they are still a few steps away from generating cash.

Indeed, picking the right success metrics isn’t easy; not because of the math part, but because of political aspects of them, too. Surely, aggressive organizations would put more weight onto metrics related to the size of footprints and the rate of expansion. More established and stable companies would put more weight on profitability and various efficiency measures. Folks on the supply side would have different ways to measure their success in comparison to sales and marketing teams that must move merchandise in the most efficient ways. If someone is dedicated to a media channel, she would care for “her” channel first, without a doubt. In fact, she might even be in direct conflicts with fellow marketers who are in charge of “other” channels. Who gets the credit for “a” sale in a multi-channel environment? That is not an analytical decision, but a business decision.

Even after an organization settles on the key metrics that they would collectively follow, there lies another challenge. How would you declare winners and losers in this numbers game?

As the title of this article indicates, you are not supposed to conclude one version of creative beat the other one in an A/B test, just because the open rate was higher for one by less than 1%. This is not some ballgame where a team becomes a winner with a walk-away homerun at the bottom of the 11th inning.

Differences in metrics should have some statistical significance to bear any meaning. When we compare heights of a classroom full of boys, will we care for differences measured in 1/10 of a millimeter? If you are building a spaceship, such differences would matter, but not when we measure the height of human beings. Conversion rates, often expressed with two decimal places, are like that, too.

I won’t get too technical about it here, but even casual decision-makers without any mathematical training should be aware of factors that determine statistical significance when it comes to marketing-related metrics.

  • Expected and Observed Measurements: If it is about open, clickthrough and conversion rates, for example, what are “typical” figures that you have observed in the past? Are they in the 10%to 20% range, or something that is measured in fractions? And of course, for the final measure, what are the actual figures of opens, clicks and conversions for A and B segments in test campaigns? And what kind of differences are we measuring here? Differences expressed in fractions or whole numbers? (Think about the height example above.)
  • Sample Size: Too often, sample sizes are too small to provide any meaningful conclusions. Marketers often hesitate to put a large number of target names in the no-contact control group, for instance, as they think that those would be missed revenue-generating opportunities (and they are, if the campaign is supposed to work). Even after committing to such tests, if the size of the control group is too small, it may not be enough to measure “small” differences in results. Size definitely matters in testing.
  • Confidence Level: How confident would you want to be: 95% or 90%? Or would an 80% confidence level be good enough for the test? Just remember that the higher the confidence level that you want, the bigger the test size must be.

If you know these basic factors, there are many online tools where you can enter some numbers and see if the result is statistically significant or not (just Google “Statistical Significance Calculator”). Most tools will ask for test and control cell sizes, conversion counts for both and minimum confidence level. The answer comes out as bluntly as: “The result is not significant and cannot be trusted.”

If you get an answer like that, please do not commit to a decision with any long-term effects. If you want to just declare a winner and finish up a campaign as soon as possible, sure, treat the result like a baseball score of a pitchers’ duel. But at least be aware that the test margin was very thin. (Tell others, too.)

Here’s some advice related to marketing success metrics:

  • Always Consider Statistical Significance and do not make any quick conclusions with insufficient test quantities, as they may not mean much. The key message here is that you should not skip the significance test step.
  • Do Not Make Tests Too Complicated. Even with just 2-dimensional tests (e.g., test of multiple segments and various creatives and subject lines), the combination of these factors may result in very small control cell sizes, in the end. You may end up making a decision based on less than five conversions in any given cell. Add other factors, such as offer or region, to the mix? You may be dealing with insignificant test sizes, even before the game starts.
  • Examine One Factor at a Time in Real-Life Situations. There are many things that may have strong influences on results, and such is life. Instead of looking at all possible combinations of segments and creatives, for example, evaluate segments and creatives separately. Ceteris paribus (“all other factors held constant,” which would never happen in reality, by the way), which segment would be the winner, when examined from one angle?
  • Test, Learn and Repeat. Like any scientific experiments, one should not jump to conclusions after one or two tests. Again, data-based marketing is a continuous loop. It should be treated as a long-term commitment, not some one-night stand.

Today’s marketers are much more fortunate in comparison to marketers of the past. We now have blazingly fast computers, data for every move that customers and prospects make, ample storage space for data, affordable analytical toolsets (often for free), and in general, more opportunities for marketers to learn about new technologies.

But even in the machine-driven world, where almost everything can be automated, please remember that it will be humans who make the final decisions. And if you repeatedly make decisions based on statistically insignificant figures, I must say that good or bad consequences are all on you.

How to Make Marketing a Revenue Center and Not a Cost

Many marketers struggle to have their departments viewed as the valuable, revenue-generating entities that they are. They are viewed as sales support teams, at best. That shouldn’t be the case, and if it is, it may be your own fault. To get your marketing viewed as a revenue center and not a cost requires the right metrics, solid sales and marketing integration, and excellent analytics.

In a word: metrics.

Many marketers struggle to have their departments viewed as the valuable, revenue-generating entities that they are. They are viewed as sales support teams, at best.

That shouldn’t be the case, and if it is, it may be your own fault. If you can’t create a line that ties your activities to actual revenue, you can’t prove that your marketing activity is generating revenue.

You only make the problem worse if you insist on talking to C-level folks about fans, followers, likes, and subscribers. They just don’t pay the rent. And acquiring them has costs.

But drawing the line between marketing and revenue isn’t always easy, particularly for B2B marketers without transactional websites. Which means that my claim for metrics being the answer is a bit simplistic. You really need:

  • The right metrics
  • Integration between sales and marketing systems
  • A measurement process

The Right Metrics

As I mentioned above, not just any metrics will do. Process metrics like fans, followers, clicks, etc. are important to us as marketers but not important to those with profit-and-loss responsibility and not important to most businesses as a whole. (If you’re in the publishing business, that’s another story.)

The metrics you need to seek out are business metrics. These are metrics related to profit, revenue, sales, lead volume, lead quality, and so on. The problem, of course, is that not only are these metrics harder to measure, they are harder to tie to specific marketing actions.

Despite the increased degree of difficulty, this is the first step in turning marketing from a cost to a revenue generator. In fact, you may have to make inroads here before you can secure the resources you’ll need to take the next step.

Sales and Marketing Systems Integration

That next step is tying your various sales and marketing systems together in such a way that you can track not just how many times a piece of content, for example, has been consumed, but what content a particular prospect has consumed. This requires coordination between your CRM system and the CMS that underpins your website.

It’s of even more value if you can track which pieces of content are most frequently consumed by prospects who convert to customers.

As an added benefit, coordination like this can also increase your marketing’s effectiveness by allowing you to tailor the content you present to individual visitors. For example, once you know what content a site visitor has already consumed, you can replace a static “You Might Also Like” links in your sidebar with links to content that might be the next logical step for someone who has already consumed introductory materials.

Progressive profiling, as it’s called, will also help you hone your content offerings and create content ladders that lead from introductory materials through education and establishing trust to, we hope, conversion.

Measurement Process

Finally, we need to measure what’s working and what is not. We need to know what content resonates with our audience and which audience segments we’re connecting with. Much of the data you’ll need for this will be available in your CRM, though you may need to tie in other analytics data gathering tools

The only downside to this is that implementation of these ideas tends to be quite customized. There’s no off-the-shelf solution that is likely to fit your needs – your audience, your CRM and CMS, your goals. Making yourself an educated consumer is critical, even if you aren’t going to implement with internal resources. Different vendors will present different solutions and doing an apples-to-apples comparison requires at least a basic understanding of the various moving parts.

Defending Your Digital Marketing Budget

It’s hard to get marketing results if you don’t have a digital marketing budget. Here are three ways to position your marketing team to get the budget you need.

It’s hard to get marketing results if you don’t have a digital marketing budget. Here are three ways to position your marketing team to get the budget you need.

1. Data Matters

First and foremost, you have to track your activity so you can demonstrate what is working and what is not. Ideally, you’ll do this in ways that make sense to the folks holding the purse strings — more on that below — but at the very least, you need to demonstrate stability and growth even if you can’t tie your activity directly to revenue.

2. Track the Right Metrics

As alluded to above, there are metrics and there are metrics. Likes, follows, friends, subscribers, are all meaningful metrics in their own way, but they aren’t bottom line metrics. We refer to these metrics as process metrics and differentiate between them and the business metrics that will be of interest to those with P&L responsibilities.

In this we’ll include lead generation, lead quality and revenue attributable to your digital marketing efforts. The more top-line difference you can make — and demonstrate — the easier the decision is to continue/increase funding for your initiatives.

Don’t forget the other side of the coin, though: costs. If you can demonstrate that the leads and revenue you generate are costing the firm less than other channels and techniques, you’ll have excellent ammunition, even if yours isn’t the biggest producer of revenue.

3. Create Value With Your Digital Marketing Budget

The overall goal is to create value for your firm. Doing this requires a data-driven approach that, at its best, can stand alone as a significant source of new business for the organization.

That can be a high bar, depending on both your industry and your organization. You can still create lasting value if your initiatives have a strong positive impact on your organization’s sales team. You’re safer if you can produce revenue, but if you can produce demand that the sales team converts, your budget is safe. (And as we noted above, if you can document that the demand you produce converts more consistently than other sources, your case becomes that much stronger.)

If you find yourself lagging behind either of these benchmarks, it may be time to proactively approach the executives responsible for your budget, swallow hard, own up to your shortcomings and seek buy-in on a plan to improve performance.

That’s no easy conversation, but it certainly beats watching your department die a slow death as it is starved of the resources it needs to be anything other than a service bureau, fulfilling requests for marketing communications materials. That’s the point at which you and your team become expendable. Creating value is an imperative for relevance — and survival.

Colleges and Universities: We Need More Focus on Marketing Metrics

First a brag: My Temple University advertising students won the Gold Collegiate ECHO — earning First Place out of 200 teams from over 30 colleges and universities. The challenge was to increase referrals for DirecTV among the existing subscriber base.

First a brag: My Temple University advertising students won the Gold Collegiate ECHO — earning First Place out of 200 teams from over 30 colleges and universities. The challenge was to increase referrals for DirecTV among the existing subscriber base.

The most telling comments from DirecTV on the winning entry:

  • “… clear understanding of the way campaigns should be analyzed, from not only response rates but offer costs and CPAs”
  • “One of the few undergrad teams with strong principles of Direct Response Marketing”

Today’s advertising and marketing students are digital natives. And while they intuitively understand digital marketing, and are even schooled in its mechanics, most don’t understand the basic metrics of acquisition cost and lifetime customer value, the key components of ROI. The Internet is a direct response medium — consumers buy things there. And with increasing proportions of marketing budgets being spent online, it’s important that colleges and universities prepare students to understand how to optimize online marketing.

According to a 2014 Gartner survey, “Digital marketing spending averaged one-quarter of the marketing budget in 2014.” Survey respondents were 315 individuals located in the U.S., Canada and the U.K. who represent organizations with more than $500 million in annual revenue across six industries: financial services, high-tech, manufacturing, media, retail and transportation, and hospitality. The survey found that “of the 51 percent of companies who plan to increase their digital marketing budget in 2015, the average increase will be 17 percent.”

But most undergraduate advertising programs focus more on traditional awareness advertising rather than response-driven advertising and the metrics that make it work. As a result, advertising students graduate without a knowledge of the key tools that will help them succeed in the in the digital marketing world.

The most gratifying part of participating in the Collegiate ECHO competition for me was seeing the students embrace direct marketing principles, like test design and acquisition cost: concepts that aren’t normally covered in traditional advertising programs. That’s one of the great things about the Collegiate ECHO competition; it provides the opportunity for students to learn these principles and apply them to a real client. By sponsoring this competition, MarketingEDGE is helping to promote education in the basic principles of direct, digital and relationship marketing — principles that will prepare students for success.

Finally, a shout-out to the winning Temple team: Bridget Doyle, Tatiana Drye, Kaitlin O’Connell and Kia Street. It was an honor for me to work with such a talented group of students. They earned the Gold with their dedication, hard work and persistence.