Question No. 1: What Kind of Response Rate Should I Expect?
There are response rate benchmark studies published by the DMA and others, usually organized by industry and type of offer (lead generation, free information, cash with order, etc.). These reports can provide you with some guidance in setting your expectations, but they can just as easily lead you astray. How? If you’ve seen one campaign, you’ve seen just that: one. But some marketers fall into the trap of applying previous results to various campaigns.
Your response rate is driven by three factors, listed here in order or importance:
- Media: If you don’t get your message in front of the right people, your response will suffer. It is the single most important driver of response, so choose wisely.
- Offer: What’s your value proposition to the prospect? Simply stated, your offer says, “Here’s what I want you to do, and here’s what you’re going to get when you do it.” If your offer is not appealing or relevant to the prospect, the response — or lack thereof — will reflect that. Also, keep in mind that soft offers, which require little commitment on the part of the prospect (e.g., get free information, download a whitepaper, etc.), will generate a higher response than hard offers, which require a greater commitment (request a demo, make an appointment with a sales rep, payment with order, etc.).
- Creative: It’s hard for traditional advertisers to believe that this element is lower in importance than the first two, but it is. And the biggest driver of response from a creative standpoint is a clearly stated prominent call to action.
Question No. 2: We Have a Strong Campaign Coming Out of Market Research. My Client/Management Wants to Get This Out As Quickly As Possible. Why Do I Have to Test?
- You may have a well-researched creative position but it can be executed in a variety of different ways (see the third bullet under Question No. 1, above). Furthermore, your market research couldn’t predict the response rates from different media. But knowing whether email lists, websites or social media fare best for your audience and offer will be crucial to generating the highest response rate.
- You want to be able to optimize the three factors above to determine which combination gives you the most qualified leads at the lowest cost per lead.
- Most importantly, you want to avoid a potentially catastrophic result if you’ve gotten one of the three key elements wrong. It’s better to do that with a small quantity rather than a full-scale effort. It’s always disconcerting to hear people say, “We tried direct. It didn’t work.” Keep in mind that if you’ve seen one, you’ve seen one. Previous successes and shortcomings won’t apply when you tweak the context.
Question No. 3. How Big Should My Test Be?
Your test should be large enough to produce statistically significant results. There are two parts to this: the confidence level of your results and the variation you’re willing to accept.
There are statistical formulas for calculating sample size, but a good rule of thumb to follow is that with 250 responses, you can be 90 percent confident that your results will vary no more than plus or minus 10 percent.
For example, if you test 25,000 emails and get a 1 percent response rate, that’s 250 responses. That means you can be 90 percent confident that (all things held equal) you will get between 0.9 percent and 1.1 percent in a rollout.
A smaller number of responses will result in a reduced confidence level or increased variance. For example, with a test size of 10,000 emails and a 1 percent response rate at a 90 percent confidence level, your variance would be 16 percent rather than 10 percent. That means you can be 90 percent confident that you’ll get between 0.84 percent and 1.16 percent response rate, with all things being held equal.