How Do You Decide What to Test in Direct Mail? 

Do you have three tested direct mail packages waiting in the wings to use when your control starts to fatigue? If you don’t, you should. It’s never a question of if a control will die, it’s when. So what is most important to test now to get to that breakthrough package? Here are some ideas.

Direct mail testDo you have three tested direct mail packages waiting in the wings to use when your control starts to fatigue? If you don’t, you should. It’s never a question of if a control will die, it’s when. So what is most important to test now to get to that breakthrough package? Here are some ideas.

Smart direct mail marketers are constantly testing. It may be the offer, positioning, format or anything else — but what variable gets you the biggest bang for the testing dollar? And which test delivers the most favorable cost per acquisition?

Traditional mail testing can be very expensive, time consuming and yield limited insight if not executed correctly.

After personally overseeing and writing multitudes of direct mail packages, it’s still tough to choose just one variable to test. The reality is that several variables should be tested all at the same time to get to a new control faster. These are the types of tests I’ve found most successful in revealing key attributes for a new control:

  • Your offer is highly influential in your response. If you’re testing price (most typical), you can test dollars off or percent off. I’ve found offering dollars off to be best, but every market is different.
  • Are you including a bonus or free gift?
  • Repositioning your product — or testing a new unique selling proposition — can reinvent your complete message and offer to produce sizeable increases in response rates.
  • A new production format can refresh an existing control. Perhaps you’ve used a #10 outer envelope for a long time. A simple switch to a #9 or #11 envelope can make a difference.
  • I like to include showstopper text and graphics on my envelopes, each worth about a half-second of time for the recipient to pause and study the OE. I’ve found elements such as faux bar codes, handstamps and seals yield favorable impact.
  • Evaluating data overlays from models or profiles will return tremendous information and insights. But if you don’t spend the time to interpret it and imagine the possibilities, you can overlook great new ideas.

So with all these test possibilities and data, what variables should you test?

In my last column, I shared a new Bayesian Analytics methodology that I think will upend direct mail testing as we know it today. Bayesian Analytics isn’t new, though its current applications are new and spreading to many fields, including weather forecasting, insurance risk management and health care policy. Later this month I’m moderating an online session on this topic (learn more at my website).

A/B testing is effective, but usually builds a new control quite slowly (how many times have you tested, only to find the test performed under your control?). Multivariate testing enables you to isolate variables and achieve a new control more quickly, but it still takes several packages to confidently identify the winner. But the use of Bayesian Analytics in direct mail gathers substantially more testing insight and produces more cost savings, while taking less than half the time of traditional testing.

I believe in taking out the guess-work of testing where it’s possible. Otherwise it is easy to incorporate our own personal emotional appeals and biases, like when we say “I’d never respond to that!” We’re probably not our own market. We’re often wrong, even as informed as we are about our products and audiences.

My point is this: You must keep testing. Test outside your comfort zone. Let your prospective customers tell you what variables they’d respond to by using Bayesian Analytics methodology to deliver the emotional insights that big data can’t deliver.

If you don’t have at least three tested packages, or knowledge of what variables form the magic combination necessary to increase response rates, Bayesian Analytics will save you a lot of time and resources.

Download my new report, “Predicting Direct Mail Results Before You Mail” to learn more about Bayesian Analytics.

Direct-Mail Testing Upended With Bayesian Analytics 

Direct-mail marketers have relied on either A/B testing or multivariate testing to evaluate winning campaigns for generations. Those evaluations, unfortunately, weren’t always based on statistics, but on educated guesses or office surveys. But a confluence of technology and something called Bayesian Analytics now enables direct mailers to pre-test and predict responses before mailing.

Direct-mail marketers have relied on either A/B testing or multivariate testing to evaluate winning campaigns for generations. Those evaluations, unfortunately, weren’t always based on statistics, but often on educated guesses or office surveys. But a confluence of technology and something called Bayesian Analytics now enables direct mailers to pre-test and predict responses accurately before mailing.

Bayesian Analytics may well upend how we test to identify the highest profit-producing control more quickly and at a fraction of the cost of traditional testing methods. Bayesian Analytics is already being used in astrophysics, weather forecasting, insurance risk management and health care policy. And now, a few cutting-edge mailers have successfully used this analytics approach, too.

Usually, direct-mail marketers test four categories of variables, such as price, headlines, imagery and formats.

Within each of those variables, direct marketers often want to test even more options. For example, you might want to test the relative effectiveness of discounts of $5 off, $10 off, 10 percent off or 15 percent off. And you want to test multiple headlines, images and formats.

The following matrix illustrates the complexity of testing multiple variables. Let’s say you want to test four different pricing offers, four headlines, four imagery graphics and four direct mail formats. Multiplying 4 x 4 x 4 x 4, you find there are a possible 256 test combinations.

GHBlog100516It’s impractical and costly to test 256 combinations. Even if your response rate dictated you only needed to mail 5,000 items per test for statistical reliability, you’d still have to mail over 1.2 million pieces of mail. If each piece costs $0.50, the total testing cost is $600,000.

Bayesian Analysis works with a fraction of the data required to power today’s machine learning and predictive analytics approaches. It delivers the same or better results in a fraction of the time. By applying Bayesian Analysis methodologies, direct mailers can make significant and statistically reliable conclusions from less data.

The International Society for Bayesian Analysis says:

“Bayesian inference remained extremely difficult to implement until the late 1980s and early 1990s when powerful computers became widely accessible and new computational methods were developed. The subsequent explosion of interest in Bayesian statistics has led not only to extensive research in Bayesian methodology but also to the use of Bayesian methods to address pressing questions in diverse application areas such as astrophysics, weather forecasting, health care policy, and criminal justice.”

Bayesian Analysis frequently produces results that are in stark contrast to our intuitive assumptions. How many times have you used your intuition to test a specific combination of variables thinking it would result in a successful direct-mail test, only to be disappointed in the results?

Bayesian Analytics methodology takes the guess-work out of what to test in a live-mailing scenario. Instead of testing and guessing (as the late Herschell Gordon Lewis wrote in his recent column, Rather Test or Guess?) you can now pre-test those 256 combinations of variables before the expense of a live mail test. The pre-test reveals which combination of variables will produce the highest response rate in the live test, resulting in substantial test savings.

But wait, there’s another benefit: You can learn what mix of variables will produce the best results for any tested demographic or psychographic group. It’s possible to learn that a certain set of variables work more successfully for people who are, for example, aged 60+, versus those aged 40-59. This means you may be able to open up new prospecting list selections that previously didn’t work for you.

Again, a handful of mailers have already pre-tested this new Bayesian Analysis methodology — it has accurately predicted the results in live testing at a 95 percent level of confidence. Now that beta testing has been completed and the methodology is proven to be reliable, look to hear more about it in the future.

There’s more about this methodology than can be shared in a single blog post. To learn more, download my report.

My new book, “Crack the Customer Mind Code” is available at the DirectMarketingIQ bookstore. Or download my free seven-step guide to help you align your messaging with how the primitive mind thinks. It’s titled “When You Need More Customers, This Is What You Do.”