Direct Mail: Why A/B Test?

This is a very common question our clients ask, especially the smaller ones. They need to know if they should be A/B testing. The easy answer is “yes.”

This is a very common question that our clients ask, especially the smaller ones. They need to know if an A/B test is something they should do. The easy answer is “yes,” because it allows you to try out different offers to see which ones work best.

Stressed, anxious person biting finger nails.If you don’t know what is working, how can you increase your ROI? Not to mention that, when you are using only one offer, what if it’s a bad one? What money are you leaving on the table by not A/B testing? Usually, the main reason people don’t want to test is because they think it will be complicated. That really is not the issue. You can create a very simple test and still gather valuable insight to apply to future direct mail projects.

Here are some ways to test your next direct mail campaign:

  • Simple test: In this test, everything about the mail pieces are exactly the same except for one element, which is usually the offer, but really you can test anything you want. You then take your mail list and split it in half, so half your list goes to one version and the other half to the second version. After you get all your responses back, it should be easy to tell which version had a better response. If you don’t have the time to collect your responses, talk to your mail provider, they can help you.
  • Control Group test: In this test, you will use a piece that you have mailed before and one test piece. You can use entirely different pieces. Decide what it is you want to test — usually design, color and images are tested in a control setting. You will split your list in half and see which one gets the better response. This works great if you are considering a new format and want to see how well it is received.
  • Complex test: In this test, you will be testing at multiple levels. This takes a lot of planning and strategy. You can test so many things — such as breaking your list of people up into like groups based on purchase history or demographics, and then creating multiple versions for each group to see what works best with each one. You can also test types of response methods from mobile texts, phone numbers, URL’s, QR Codes, reply cards and more.

One of the great results with testing is what you learn each time. It is extremely important to add this information to your database so you have it available for future testing. The more you know about your prospects and customers, the better your direct mail offers will be. Basically, A/B testing is a waste of time if you are not tracking your results. So let’s look at ways to do that.

Here are some ways to track direct mail results:

  • Unique phone number: Create a special phone number for each campaign.
  • Unique URL: Create a special landing page URL for each campaign.
  • Unique coupon code: Create a special code for people to use at purchase for each campaign.
  • Unique short code: Create a special texting short code for each campaign.

Now that you have tracking information, you can analyze it to create reports. You want to know which version had the best response, the most spend, who responded to what, and so on. The most important thing to know is what your ROI was on each version. Some things we do cost more money than others, so knowing if spending more gets you more money is vital otherwise you could just be wasting your money.

Have you tried testing? What has worked best for you?

Author: Summer Gould

A blog about Direct Mail Marketing, tips, tricks and what not to do.Summer Gould is President of Eye/Comm Inc. Summer has spent her 27 year career helping clients achieve better marketing results. She has served as a panel speaker for the Association of Marketing Service Providers conferences. She is active in several industry organizations and she is a board member for Printing Industries Association San Diego, as well as a board member for Mailing Systems Management Association of San Diego. You can find her at Eye/Comm Inc’s website:, email:, on LinkedIn, or on Twitter @sumgould.

4 thoughts on “Direct Mail: Why A/B Test?”

Leave a Reply

Your email address will not be published. Required fields are marked *