Marketers, How Valid Is Your Test? Hint: It’s Not About the Sample Size

The validity of a test is not tied to the size of the sample itself, but rather to the number of responses that you get from that sample. Choose sample sizes based on your expected response rate, not from tradition, your gut or convenience.

A client I worked with years ago kept fastidious records of test results that involved offers, lists, and creative executions. At the outset of our relationship, the client shared the results of these direct mail campaigns and the corresponding decisions that were made based on those results. The usual response rates were in the 0.3 to 0.5% range, and the test sample sizes were always 25,000. If a particular test cell got 130 responses (0.52%), it was deemed to have beaten a test cell that received 110 responses (0.44%). Make sense? Intuitively, yes. Statistically, no.

In fact, those two cells are statistically equal. With a sample size of 25,000 a 0.5% response rate, your results can vary by as much as 14.7% at a 90% confidence level. That means that there was a 90% chance that the results from that test could have been as much 0.55% or as little as 0.43%, making our test cell results of 110 responses (0.44%) and 130 responses (0.52%) statistically equal. I had to gently encourage the client to consider retesting at larger sample sizes.

There are statistical formulas for calculating sample size, but a good rule of thumb to follow is that with 250 responses, you can be 90% confident that your results will vary no more than +10%. This rule of thumb is valid in any medium online or offline. For example, if you test 25,000 emails and you get a 1% response rate, that’s 250 responses. Similarly, if you buy 250,000 impressions for an online ad and you get a 0.1% response rate, you get 250 responses. That means you can be 90% confident that (all things held equal) you will get between 0.9% and 1.1% in the email rollout,  and between 0.009% and 0.01%, with a continuation of the same ad in the same media. (Older editions of Ed Nash’sDirect Marketing — Strategy, Planning, Execution contain charts that you can reference at different sample sizes and response rates).

A smaller number of responses will result in a reduced confidence level or increased variance. For example, with a test size of 10,000 emails and a 1% response rate (100 responses), your variance at a 90% confidence level would be 16%, rather than 10%. That means you can be 90% confident that you’ll get between 0.84% and 1.16% response rate  with all things being held equal. Any response within that range could have been the result of variation within the sample.

Marketers are not alone in using their gut rather than statistics to determine sample sizes. Nobel Laureate Daniel Kahneman confesses in his book “Thinking, Fast and Slow“:

“Like many research psychologists, I had routinely chosen samples that were too small and had often obtained results that made no sense … the odd results were actually artifacts of my research method. My mistake, was particularly embarrassing because I taught statistics and knew how to compute the sample size that would reduce the risk to an acceptable level. But I had never chosen a sample size by computation. Like my colleagues, I had trusted tradition and my intuition in planning my experiments and had never thought seriously about the issue.”

The most important takeaway here is that the validity of a test is not tied to the size of the sample itself, but rather to the number of responses that you get from that sample. Choose sample sizes based on your expected response rate, not from tradition, your gut or convenience.

Consumer Marketers, Looking to Test New Data Categories? Try These

We are all trying to create and sustain customers, using data to discover new patterns, new audiences, and new prospects — and that requires a lot of testing, and innovative data sets to explore (responsibly). Let’s make it experiential, as well as experimental.

We in the data marketing business love to test — at least, we should. And what we should test for is new data categories.

Expanding the marketing universe — and stretching the marketing budget — depends on higher efficiency in our lists, offers, and creative. We should be eager to test new proofs of concepts and new categories of data sources as they enter the market … if only to know whether or not they produce incrementally or otherwise.

I’m still surprised when I hear some of my data-vendor friends say that a good number of their clients pass on testing — and just go all-in on new lists and data sources. It seems like testing is still too much work for some, or they feel the only way to test is with an entire data source. Guess these client-side folks have money to burn, or are operating very much on-the-fly.

In some ways, digital marketers have it all over offline marketers in their ability to test, cycle, test again, and so on — often, many times over by the time a direct mail or direct-response print or broadcast test cycle has run its course. Yet, in this speed, have we sacrificed some quality in our prospecting strategies?

Online audience algorithms can produce some highly categorized niche segments, based on site visits and app usage — much of it de-identified, from a personal perspective. But how do these segments really stack up against a transaction database, or response lists, or even compiled lists, based on personally identifiable information? Thankfully, we can test for this, or even overlay data! (I am not advocating re-identification here, nor should you. Oh California, please don’t force us to identify non-PII. It’s soooo anti-privacy.)

Recently, the Direct Marketing Club of New York (DMCNY) held a very interesting breakfast program titled “Beyond Demographics: The Data You Need to Max Out Marketing Performance.”

Some Fresh Categories for New Reach and Affinity Discovery

Consider some of these data sources for testing:

  • Values Data — Test cohorts based on “shared values,” rather than simply choosing audiences based on demographics or psychographics. David Allison, principal, David Allison Inc., and author of “We Are All the Same Age Now,” pointed to his firm’s internal research that shows that popularly defined age groups rarely (or barely) match on what they agree upon, or value, as a generation. For example, Baby Boomers agree with each other about 13% of the time; Gen X, about 11% of the time; and Millennials, 15% of the time. Thus, targeting based on demographics alone can be extremely wasteful if the marketer is assuming some sort of shared attribute among them, other than age.However, when targeting based on shared “values” — Adventurers, Savers, and Techsters, and the like — all of a sudden affinities jump sky-high. In these cases, 89%, 76%, and 81%, respectively. These “valuegraphics” are based on “big data” segments — rather than small data (response lists, for example). Still, when compared to demographics targeting alone, shared-value targeting offers an eight-time lift!  Well, that’s worth testing.
  • Attitudinal Data — Another perspective on “beyond demographics” came from Mark Himmelsbach, co-founder, Episode Four, a creator of “brand hits,” such as this one for Charles Schwab. We often have stereotypical views of many demographic and other audience categories — and too many algorithms, he said. But analyze the data for unusual patterns, and suddenly you can find “who knew?” commonalities among certain audience segments that would wow any of us.Who knew that ultra-high net worth individuals are electronic dance music enthusiasts? Who knew that African-American married women are high on the e-sports genre? Or that young Hispanic/Latino adventurers are really into escape rooms? These discoveries give brands new advertising, product placement, and sponsorship opportunities, for example, which might otherwise go untapped. I’m still trying to get my head around these reported affinities, based no doubt by my own preconceptions.
  • Location Data — According to the World Economic Forum, 90% of the world will soon have or already has a supercomputer in their pocket — a smartphone. We’re actually closing in on four connected devices per person, reports Jeff White, founder and CEO, Gravy Analytics. With smartphones alone, as constant companions, we have a huge opportunity to leverage responsibly use of location data. Location can provide huge “affinity” targeting opportunities.A casual wine user might search and buy online his or her wine. But a wine aficionado visits a winery (Location X), or attends a wine tasting (Location Y), and now you have a true affinity opportunity. Granted, location data has a level of sensitivity that carries, more often than not, an opt-in requirement — but the marketing lift can be a significant reward for the advertiser who strategically applies such insights from it. Makes me want to tag every latitude and longitude for some hobby or interest!
  • Experiential Data — Live Nation may own concert venues, Ticketmaster, online game communities and music/culture festivals — but across these many first-party experiences, the company can provide deep analytics that help monetize its various audiences through enriched second-party relationships, said Anubhav Mehrotra, VP, Live Nation. Hilton, American Express, and Uber are just some of the brands Live Nation has teamed up with to enrich brand users with engaging experiences, such as backstage tours and “meet the artists.”

We are all trying to create and sustain customers, using data to discover new patterns, new audiences, and new prospects — and that requires a lot of testing, and innovative data sets to explore (responsibly). Let’s make it experiential, as well as experimental: I sure hope to meet some ultra-high-net-worth individuals at the next Electronic Dance Festival I attend. Or not.

Toasting 2018 Silver Apple Honorees: In Their Words

You might have heard of a big event that happened last week in the USA. No, not THAT one. I’m talking about but the presentation of the Direct Marketing Club of New York’s 2018 Silver Apples honors. Here’s more about the awards, from the Silver Apple honorees themselves.

Silver Apple Honorees ballroom
Photo Credit: Edison Ballroom via DMCNY, 2018

You might have heard of a big event that happened last week in the USA. No, not THAT one. I’m talking about but the presentation of the Direct Marketing Club of New York’s 2018 Silver Apples honors. Here’s more about the awards, from the Silver Apple honorees themselves.

The Silver Apples recognize leadership, stewardship and business success mid-career in the data, direct and digital marketing field. Each honoree has (more or less) 25 years of experience, with matching achievements to point to … and all have additional contributions to our industry, community, mentoring and giving back.

With the assistance of newly named The Drum U.S. Editor Ginger Conlon, I thought it worth amplifying a few key industry insights shared by this year’s individual honorees:

Anita Absey, Chief Revenue Officer, Voxy (New York):

Favorite Data Story: “Back in the very early days when I was at Infobase, we were doing data overlays on customer databases, which was novel at the time. While working with a large insurer, doing overlays of demographic and socioeconomic data on their database, the profile and segmentation scheme that emerged from that work actually defied some of the assumptions that they had about the characteristics for their customers’ profile. The insights we provided them helped them make subtle changes in their communications and targeting to customers, which improved the overall risk profile of their customer base. It was gratifying to see how data could affirm or deny assumptions and enable our client to make decisions that helped improve the risk profile of their business.”

Measurement: “Hope is not a strategy. Your actions have to be data-based, not hopeful. Similarly, you can’t manage what you can’t measure. Unless you have data that points you to the actions and decisions that are best for the business, you’re running blind.”

Matt Blumberg, Co-Founder and Chief Executive Officer, Return Path, Inc. (New York):

On Choosing Marketing: “The thing that drew me to marketing was the Internet. I had been working as an investor at a venture capital firm that invested in software companies. Once Netscape went public and people started figuring out the short- and the long-term potential of the Internet, I got very excited about working in that field. Unbeknownst to me at the time, the Internet is all about direct marketing. For the first several years of my career, I would never have described myself as a direct marketer; but in hindsight, obviously, I was.”

On Inspiration: “It’s several sentences out of a speech by Theodore Roosevelt called ‘The Man in the Arena.’
It’s incredible. It goes:

” ‘ … The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.’

“I take it as the entrepreneur’s motto. It’s a beautiful passage that I have taped up everywhere.”

Pam Haas, Account Director, Experian Marketing Services (Providence, RI):

Overhyped: “Display and programmatic technologies are overhyped. It’s like the early days of email marketing: People just started sending millions of emails, hoping some would stick. The same thing is happening in display and programmatic. That part of the industry still needs to mature.”

Best Metric: “Right now, it’s the ROAS: Return on Ads Spent. I love that. For every dollar that the client is spending, we know that we are driving X number of dollars in sales.”

Career Advice: “Diversify. In marketing, there are so many different angles and specialties that you can focus your career on. Throughout my career, I’ve [been] able to gain experience in multiple facets of marketing: direct response, email technology, and in databases and modeling. Digital is so sexy right now, but the fundamentals still apply; so it’s important not to pigeonhole yourself into one area.

“While in a mentor program at Equifax, my mentor was a woman and she told me, ‘You have to be your own PR person. You have to make your accomplishments known, because nobody else is going to do that for you.’”

Keira Krausz, EVP and CMO, Nutrisystem, Inc. (Fort Washington, Penn.):

On Her Current Assignment: “I’m proud of where we are at Nutrisystem and I’m particularly proud of what we’ve built as a team. Our job is wonderful, because we get to help people live healthier and happier lives. Since 2013, we’ve nearly doubled the business, which means we’ve helped a whole lot more people get healthier and happier. Along the way, we’ve revamped nearly every aspect of our business that you can think of, and we’re just getting started.”

On Mentoring: “In my first years in marketing, I was always being asked what my goals were and how I saw myself in years to come, and I always felt flummoxed, because I didn’t know what to say. I wasn’t one of those young people who had their whole life planned out when I was 25, and I often felt insecure about that. But it turns out that was OK.

“So, one thing I did that I would advise is, from early on, try to work for someone you can learn from. Somebody who you admire, who has something unique, and who can teach you something that you think you’re missing. The rest will fall together.”

Tim Suther, SVP and General Manager, Data Solutions, Change Healthcare (Lombard, Ill.):

On Career Choice: “I’ve always been technology-oriented, from learning to code when I was 17 to graduating college with a finance degree. With that background, naturally, I was suspicious of marketing. A lot of marketing felt inauthentic and superficial to me. But I had this one moment where I actually saw a dynamic gains curve for the first time and I thought, ‘Oh my god, this is one of the most interesting things I’ve ever seen.’ It was the intersection of the art of marketing and the science of data that really drew me in; and boy, did I get lucky on that one, because that’s what it’s all about today.”

On Being Data-Driven: “This might surprise you a little bit, but it annoys me when marketers say that they’re data-driven, because that’s like saying, ‘OK, it’s time to turn off my brain and just let the data drive the story.’

“I think marketers are far better off when they are data-informed, where they’re combining what the data is telling them with their own business judgment to make the right decision. Human behavior is still too complicated to purely reduce to what an algorithm tells you to do; it has to be a combination of what the data is saying, creative savvy and business judgment.”

This year, DMCNY added two special awards not tied to mid-career, but recognizing two huge drivers in our business today: advocacy and disruption. The inaugural Apples of Excellence 2018 honorees include:

Advocacy:

Stu Ingis, Chairman, Venable LLP (Washington, D.C.):

On Policy-Making: “The whole privacy concern is overhyped. What’s not getting its fair recognition, in the policy world, is all of the innovation that the marketing community brings to society. For instance, they’re bringing real-time targeted marketing to television and delivering marketing communications that consumers are interested in on a personalized basis.”

On Careers: “Take the long view. Work really hard; don’t worry about the compensation or the glory, and then persevere. Stay with it. Don’t switch jobs all the time thinking that something else is always better. If you develop your skills, the good work will come to you. You don’t have to go to it.

“I’d been representing the DMA for about two years, and I had an opportunity to leave the law firm and go out in the early Internet age at Yahoo!

“Yahoo! stock was going up. I would have made millions of dollars a day. I went to Ron Plesser and said, ‘I like working for you; I like the clients; I like the work I’m doing. But I could go get really rich working for this company.’ He said, ‘Why do you want to do that? It’ll ruin your life.’ For whatever reason, I actually believed him and agreed with him. And I stayed at my job. It was probably the best decision I ever made. I don’t regret it for a second.”

Disruptor Award, Presented by Alliant:

Bonin Bough, Founder and Chief Growth Officer, Bonin Ventures (New York):

About Bonin: “His unique approach of applying innovative technology to create breakthrough campaigns helped to reinvigorate traditional marketing brands, such as Gatorade, Honey Maid, Oreo and Pepsi.

“But his influence doesn’t stop there. Bonin believes in supporting young talent and savvy entrepreneurs. While at Mondelēz International, for example, he created internal programs to mentor young talent and launched a startup innovation program, Mobile Futures, to provide a platform for marketing-tech and agency start-ups to work with the CPG giant.

“Stephanie Agresta, global director of enterprise growth at Qnary, describes him best in her recommendation on LinkedIn: ‘Bonin is a force of nature … A true rockstar from Cleveland to Cannes, Bonin has been [at] the forefront of the digital revolution from the beginning. Smart, successful, and connected, Bonin has the pulse on what’s next. Those that know Bonin well can also attest to his generosity, commitment to mentorship and a deep belief that anything is possible.’”

Since I had the privilege of interacting with Bonin at DMA &Then18 recently, I can attest the walls fall away when you converse with him. Disrupted, indeed.

All of these honorees as well as corporate recipient Winterberry Group have many things to teach us. That’s why it’s important we continue to recognize these business leaders, as marketing today, as Matt Blumberg says, is a 100 different things. It’s the business outcomes that matter.

3 DRTV Testing Tips for Digital Marketers

Lately, I have been talking to several marketers that want to test direct response television advertising for their brands. Interestingly, these companies that want to test DRTV are category disruptors, born from the Internet. These are companies founded on direct relationships with consumers established through search engines and social media.

DRTVLately, I have been talking to several marketers that want to test direct response television advertising for their brands. Interestingly, these companies that want to test DRTV are category disruptors, born from the Internet. These are companies founded on direct relationships with consumers established through search engines and social media.

The reasons to complement data-driven digital marketing with television are convincing. These internet brands are faced with many challengers in the same space. It is difficult to establish a unique brand position through search. Television remains the most powerful medium for quickly communicating a message and establishing an identity.

More importantly, marketers are finding that building companies one-click at a time is not achieving their goals for growth. Nielsen reports the average American still watches 5 hours a day. With that large an audience television can quickly scale the reach needed to grow businesses.

Most companies are concerned with the high-entry costs for television and not getting the instant ROI expected from digital channels. That’s what makes the more accountable DRTV so compelling. The opportunity to lower media costs with the assumption that the spots will generate response.

These expectations for DRTV might be misunderstood and a little high. One of the companies that I spoke with did a test of television (on their own and not with my direction). They ended up canceling after 2-weeks because of cost and perceived lack of performance.

Before testing television’s impact on response and acquisition, ensure your test is set-up to deliver effective and measurable results. A poor test could result in mistakenly dismissing a potentially valuable channel for marketing.

Here are a few tips for an effective test:

Tip #1 – Utilize Television to Its Strength – Reach

Television is a mass medium and is most efficient when used to reach a mass audience.

Digital marketers might believe that an impression delivered to a non-targeted prospect is useless and a wasted expense.  Restricting reach to select areas using cable or specific homes through addressable OTT may seem to solutions for waste. However, these options reduce consumer reach while increasing media costs, impairing the benefits of television, efficiency and scale.

With your initial television test, utilize the networks, stations, and dayparts that best reach your target audience.  If the results are good, you can start looking for ways to optimize your buy to drive more efficiency. If the television doesn’t drive results, at least you don’t need to second guess your media execution.

When putting together a test with the right stations and dayparts, be less concerned about Reach, Frequency, and GRPs.  Those are metrics to predict the effectiveness of a traditional television buy. If you are testing DRTV with the intent of generating response, results should be the measure of effectiveness.  Do look for deals as a lower cost per spot should contribute to a lower cost per response.

Tip #2 – Track All Results!

My preferred method for tracking the performance of a DRTV test are dedicated phone numbers assigned to each station.  With time-stamped, phone number data it is possible to identify the best stations and dayparts for driving results.

However, a digital disruptor may not be set-up for phone calls. They will likely want to drive response to its online sales-funnel.  Spot airing times and web-traffic data can be aligned to measure direct response to the ads. Establish site traffic baselines before spots begin airing to quantify traffic lift in response to television.

When tracking all results, also look for lift in response rates to other channels. Unlike other mediums, the awareness created by television is likely to increase response in other channels.

Quantifying the impact television can have on other channels can be done with the classic tactic of Control versus Treated. By holding out a market from a recent program, I had comparison data to quantify the lift DRTV had on other channels. The test showed a 25% increase to the direct mail response and a 50% lift in the search clickthrough rate.

If a control hold-out isn’t possible, compare response rates from pre and post television advertising. The lift in response rates can be enough to support using television as a compliment to other direct channels.

Tip #3 – Give DRTV Testing Some Time

Testing television for the first time is big decision.  It requires company buy-in and investments of time on money to develop a spot and get it on television. The desire to show immediate success is great, however, it is a test. Results can take time to develop.

It can take a couple of weeks for a brand with limited awareness to connect with consumers.

Two-weeks of frequency may be needed to build enough interest to elicit a response. It can take longer depending on your DRTV schedule.

If results are limited after two-weeks consider adjusting the television schedule to test different programming.

At 4-weeks you should have quantified results, either direct response to the television or response lift in other channels. With results you can begin assessing the opportunities for expanding the program.

If the results are not seen in 4-weeks, then it might be time to suspend the program. As direct marketers know, more spending isn’t likely to improve ROI.

Because the initial test didn’t produce the desired results does not mean that television cannot work for your brand. Go back and reconsider all elements of the program including the spot, the offers, and the media buy.

Implement DevOps to Improve CX

There’s been a movement in IT during the past seven or so years to adopt a DevOps methodology; whereby, developers and operations are working together to deliver sound secure code and applications in a frequent and timely manner.

There’s been a movement in IT during the past seven or so years to adopt a DevOps methodology; whereby, developers and operations are working together to deliver sound secure code and applications in a frequent and timely manner. Companies have gone from releasing new code annually or semi-annually to continuously in order to meet customer needs. Software companies are focused on delivering a great user and customer experience (CX).

The success of DevOps got me to thinking, “Will the same methodology work for sales, marketing and customer service?”

The steps are straightforward and intuitive:

  1. Plan
  2. Create
  3. Verify
  4. Package
  5. Release
  6. Deploy
  7. Monitor

Let’s look at each step in the process and see how it might work for sales, marketing and customer service to improve customer experience.

  • Plan: Where every project needs to start. All three parts of the business need to sit down together and define the business problem they’re trying to solve. Given that 85 percent of companies think they’re delivering a good customer experience, while only 15 percent of customers believe the same indicates there’s a huge gap between perception and reality. Perhaps you need an NPS or customer satisfaction study to help everyone get on the same page about the problem(s) you need to solve to improve the customer experience you are providing.
  • Create: Here’s where you need to map the customer journey from initial consideration to installation and repurchase. You also need to know what the customer expectation and experience are at each of the stages of the customer journey. You can create a hypothetical customer journey by collecting the experiences and impressions of your team, as well as analyzing all of the data you have regarding awareness, attribution and survey results.
  • Verify: You need to validate the accuracy of your hypothetical customer journey map with your customers. Data can tell you a lot. Real customers can tell you a lot more. How accurate is your map? At what step in the customer journey are you meeting customer expectations, where are you exceeding them and where are you failing to deliver? Talk to customers to find out.
  • Package: After you’ve verified your customer journey map, it’s time to identify steps to take to improve the customer experience. You may identify a dozen or more opportunities; however, start small. Have sales, marketing and customer service each identify one thing they can add to or change in their current process to improve the customer experience. As you have success with those initiatives, and see the positive results, you can take on more initiatives.
  • Release: Start doing the three things you identified with a segment of your audience. Sales may be differentiating marketing-qualified leads from sales-qualified leads. Marketing may be providing more personalized, relevant information of value. While customer service may be using a 360-degree view of the customer so they already know what the customer’s issue is and are able to resolve it on the first call (or email or text).
  • Deploy: Once the release is complete and you know how the initiatives are performing, you can deploy the initiatives across your entire audience of customers and prospects.
  • Monitor: Perhaps the greatest return on the DevOps process is the speed at which the organization learning about how its applications are performing. Short feedback loops let the DevOps teams know how consumers are responding to their apps and their code and improvements can be made quickly and released back to the consumer who sees the continuous improvement. This can be a tremendous benefit for sales, marketing and customer service.

Sales sees productivity increase and sales cycles shorten as they focus their efforts on sales-qualified leads. Marketing sees greater open and click-through rates with more relevant communications. Customer service sees and hears happier customers who are getting their questions and problems resolved more quickly.

Now that these three initiatives have been implemented, you can tackle the other five, 10 or 20. Improving the customer experience is a never-ending journey, but one which differentiates your company from your competition, while generating more revenue, more repeat purchases and more customer equity.

Testing: Sometimes, the Little Things Count

We’re always seeking new approaches to achieve breakthrough results in direct response marketing, and sometimes we get a big win with a bold new concept. But while you’re on the quest for a big idea, don’t overlook the little things. Slight changes, particularly in your call to action or offer structure, can result in incremental improvements and even significant lifts in response. And in the online world, these tests are easy to execute and results can be read quickly.

testing in marketing
“A/B testing,” Creative Commons license. | Credit: Flickr by Howard Lake

We’re always seeking new approaches to achieve breakthrough results in direct response marketing, and sometimes we get a big win with a bold new concept. But while you’re on the quest for a big idea, don’t overlook the little things. Slight changes, particularly in your call to action or offer structure, can result in incremental improvements and even significant lifts in response. And in the online world, these tests are easy to execute and results can be read quickly.

Recently, I moderated a panel for the Philly DMA. Panelists Caleb Freeman of Calcium USA, Kate Gomulka of MayoSeitz Media, Julie Herbster of Quattro Direct, and Arly Iampietro of Nutrisystem all shared tips on optimizing online marketing.

Some of the insights Iampietro conveyed illustrate how little things can result in significant wins. She’s responsible for optimizing the website user experience at Nutrisystem. So Iampietro showed us how minor changes to the copy on the call-to-action button, like using a caret (>) next to the button text, made a big difference in guiding customers through the conversion process. Also, for a high-commitment product like a diet program, things that suggested a commitment by the visitor before the person was ready to commit depress response.

For example, early on in the user experience:

  • “Continue >” stimulates more clicks than “Order now >”
  • “View plan >” outperforms “Get started >”

Other easy tests include changing up the imagery on the site. For example, men converted better when shown pictures of men with women rather than with other men.

Offer structure is highly important and easy to test. In his book “Predictably Irrational,” psychologist Dan Ariely relates a classic example of how a simple change in the offer structure made a difference for The Economist magazine. He presented a group of 100 MBA students with the following offers:

  • Internet-Only Subscription                    $59
  • Print-Only Subscription                        $125
  • Print and Internet Subscription          $125

The results — 84 selected the combo offer vs. 16 selecting Internet only. With the “Print Only” decoy offer removed, only 32 selected the combo offer. I repeated this experiment with a group of 30 undergrad students, presenting the scenario without the decoy offer first. None of them chose the print and Internet option. With the decoy offer inserted, about half went for the combo offer — and these were budget-conscious undergrads who don’t consume print as a rule!

Easy-to-execute tests that bring big results are not limited to online marketing. In my agency days, we had a B2B client who was sending millions of mailers out each season. We tested multiple creative executions against a very recalcitrant control every year, without a significant win. Then we decided to buck traditional direct wisdom that mailing to an actual name would outperform a non-personal address. What we found was that addressing mail to the functional title of the decision-maker outperformed our mailings to specific names, some of which may no longer have been at the company. So don’t be afraid to test against your long-held assumptions, so long as you do it in a limited way.
The takeaway? A big win doesn’t always come from a big change.

What little things have you done that have generated big wins?

Embrace Failure to Achieve Success

Too many marketers fear failure instead of embracing it. They fear that reporting poor results will be viewed as poor management. Instead, they should be positioning their results as learnings. Knowing what doesn’t work is just as important as knowing what does; yet the fear of failure permeates many corporate cultures, discouraging risk-taking and encouraging the status quo.

failure
(Image via iskandariah.perubatan.org)

Too many marketers fear failure instead of embracing it. They fear that reporting poor results will be viewed as poor management. Instead, they should be positioning their results as learnings. Knowing what doesn’t work is just as important as knowing what does; yet the fear of failure permeates many corporate cultures, discouraging risk-taking and encouraging the status quo.

There have been many times when I proposed a limited test plan with a small downside only to have it rejected by the client in favor of “the way we’ve always done it.” Following the course that nobody ever got fired for may be the politically safe option, but breakthrough results are never achieved from the status quo. As Theodore Roosevelt said, “The only man who never makes a mistake is the man who never does anything.”

Reporting on the acquisition of Whole Foods by Amazon, The New York Times noted, “While other companies dread making colossal mistakes, Mr. Bezos seems just not to care … That breeds a fiercely experimental culture that is disrupting entertainment, technology and especially retail.” (June 18, 2017) Commenting on Bezos’s style, Farhad Manjoo said in his column State of the Art, “The other thing to know about Mr. Bezos is that he is a committed experimentalist. His main way of deciding what Amazon should do next is to try stuff out, see what works, and do more of that.” (NYTimes June 19, 2017) Something direct marketers have done for decades.

Learning to embrace failure is an acquired skill. Smith College has instituted a new program called “Failing Well” to destigmatize failure for the high achievers who are admitted to the prestigious school on the basis of their perfect resumes. Smith’s Rachel Simmons says, “What we’re trying to teach is that failure is not a bug of learning, it’s a feature.” (NYTimes, June 25, 2017)

David Ogilvy, a strong proponent of testing and measurement, addresses the importance of embracing failures in the Ogilvy on Advertising chapter entitled “The 18 Miracles of Research.” He relates a story about a client who had invested $600,000 (a large sum in Ogilvy’s day) to develop a new product line. Ogilivy says, “ … our research showed a notable lack of enthusiasm … When I reported this discouraging news to the client I was afraid that, like most executives faced with inconvenient research, he would argue the methodology. I underestimated him. ‘Dry hole,’ said he, and left the meeting.”

Testing and experimentation is easy in the digital marketing environment. Even the best-conceived test plans will produce more failures than successes. Embrace those failures as valuable learnings.

My Best Short-List of Digital Sales Prospecting Tools

What tools do most of my best (most productive) students use when prospecting? Glad you asked. Here’s the skinny on “top picks” that have been crash-tested by me, my team and my customers … soloprenurs and sales reps from large corporate teams.

Email, Mobile and Social Media Marketing: Lessons from top-performing B-to-B and B-to-C brandsWhat tools do most of my best (most productive) students use when prospecting? Glad you asked. Here’s the skinny on “top picks” that have been crash-tested by me, my team and my customers … soloprenurs and sales reps from large corporate teams.

These aren’t the coolest tools. Instead, these are vetted products and services that seem to be used by top-performing sales people and entrepreneurs. Do you and/or your team work virtually? Get ready for the goods.

Email Open/Download Tracking and Scheduling

Don’t know your email open rate when sending cold prospecting messages? You’re flying blind. It’s impossible to judge effectiveness of your message without first knowing your subject line is effective.

If you aren’t being opened, your message isn’t getting read. Many of my students report 40 percent and even 75 percent open rates. Sound impossible? It’s not. But you’ve got to get out-of-the-box, get creative … take risks.

To understand email open rates, Mac (Apple Mail) users seem to prefer MailButler. For non-Mac heads, Boomerang for Gmail is a strong option, especially for businesses who allow Google to host their email via G-Suite.

MixMax and SalesHandy are two newcomers who are pushing beyond the usual functionality of Gmail/G-Suite plugins. My team and I use MixMax. These tools offer a wide (and growing) array of simple functions vital to your digital prospecting success. These include email:

  • Reminders (return to top of inbox on certain date based on recipient action taken)
  • Scheduling of message delivery at specified time
  • Template storage and message insertion
  • Open tracking and notifications (showing real-time, location and device type)
  • Download notifications (showing when prospects open your attachments and links)
  • Message sequencing (based on a pre-defined or on-the-fly cadence you use)

Need to integrate with your CRM of choice? No problem. Most of these solutions stand ready to help you move contacts to various popular CRM tools. Many SaaS CRM tools are also able to “suck in” email message data from your cloud G-Suite account.

This kind of practical reporting (open and response rate) makes weekly reporting to the boss (or yourself) a snap.

Email Verification Tools

When prospecting, we’re forced to email prospects. LinkedIn InMail alone is not enough — contrary to what most folks think. There are various, reliable hacks and methods to identify most business email addresses. However, most are a chronic waste of time.

Frankly most free email verification tools (and CRM platforms that include access to contact data/emails) are horrific. Quality lacks. Most generate 30 percetn or less accuracy rates on valid email addresses. This is what I and my students experience.

However, there are a few that seem to shine at understanding which business email address is valid and which is not.

Among the better choices (based on collective experience … please share yours too!) are:

  • Proofy.io (free and affordable paid email verification)
  • Emailvalidator.co (free email validation)
  • Hunter.io (free email validation)

Digital Calling and Telephone Tools

Whether you’re a small business entrepreneur or part of a global sales team you need to be on the phone. Like it or not, cold calling remains essential to success of 85 percent of students I coach.

Thus, you may need (or benefit from) emerging VOIP telephone tools like Grasshopper.com. This service makes it easy for small organizations to look-and-feel big. Once you have a registered toll-free phone number the possibilities begin. Your business can have:

  • Multiple extensions: an automated call routing system (even if there are only 2 of you!)
  • Voicemails transcribed and emailed or texted to you
  • Call forwarding
  • Business text messages

Hardware required? Zero. It’s all in the cloud.

Microsoft-owned Skype should not be overlooked, especially if you’re like me: traveling and working with international clients. Simply secure your phone number and bring it with you no matter where on the planet you are — so long as you have Wi-Fi, you have your local phone number.

Skype Manager is a great team management tool complete with call reporting for your in-office or remote team.

But don’t overlook UpCall.com. If you’re a business owner or manager interested in finding high-quality, vetted cold-callers, this service is for you.

In essence, UpCall.com is “like UpWork (formerly Elance) for cold-callers.” This is a growing database of cold-callers from around the globe. Select language, geographic and other preferences you require for callers. Then UpCall’s search engine suggests callers that best fit your requirements.

You can even listen to successful calls the vendors have made, view training and certifications they hold, and more.

Best of all, callers are rated based on feedback of paying clients they call for. Just like on Ebay or UpWork, you can choose the “safest bet” to start a new vendor relationship.

These are just a few telephone and email tools I have come to use, recommend based on my (and my customers’) experience. What are your experiences and preferred tools? Do they differ? Please share in comments!

How Not to Be a Tone-Deaf Brand

Pepsi is clearly not the only brand to take a wrong turn in their ads or other communications. A few recent examples include Match.com, Nivea, and most recently, United Airlines. The Internet is poised to identify, amplify and vilify brands that make these kinds of mistakes. So, how can a brand avoid being tone deaf?

A lot of news sources have already unpacked and dissected the many missteps in the recent Kendall Jenner Pepsi ad. It was live for all of one day, but it will live on in brand infamy forever. Here are a few of the sources covering the cringe-worthy effort, if you want to dig deeper into this example:

Nivea "White Is Purity"
The official Nivea page on Facebook is putting out ‘pro skin whitening’ adverts with the slogan ‘white is purity’
Source: Facebook

Pepsi is clearly not the only brand to take this kind of wrong turn in their ads or other communications. A few recent examples include Match.com’s misguided characterization of red hair, freckles, or eye color as imperfections, as well as Nivea’s portrayal of “White is Purity” that earned immediate online fury. Most recently, United Airlines compounded their operational, planning and basic humanity missteps with a decidedly tone-deaf CEO communication that instead of calming the public outcry further fueled it.

The Internet is poised to identify, amplify and vilify brands that make these kinds of mistakes. Meanwhile consumers marvel at how, in this day and age, brand and creative strategists make the poor decisions that pass the many layers of consideration and reviews of these expensive efforts. It’s a fair question.

So, how can a brand avoid being tone deaf?

The Brief

If your ad vision and subsequent brief’s objective is to tune into recent global trends or to otherwise tack onto complex societal issues, go back to the drawing board. You are not qualified to establish world peace, nor is it your brand mission. You can brag a bit about a sincere and established philanthropic effort or partnership, but that right is only earned after you have had a real world impact. Better yet, promote the effort, rather than the brand, and let the public laud you indirectly. The halo effect is very flattering.

If you have doubts about an approach or messaging platform under consideration, now is the time to trust your gut and voice those doubts. Spend some time and resources testing consumer reaction to get a direct read on how your vision could be perceived.

Hype or Opportunity? 

Marketers today face the huge challenge of creating the right program mix to meet their brand objectives. It’s difficult to balance the risk of new investments against the budget support needed to continue in proven channels. But it could be even riskier to wait too long to test or adopt some of the newer opportunities that emerge with oppressive regularity.

Marketers today face the huge challenge of creating the right program mix to meet their brand objectives. It’s difficult to balance the risk of new investments against the budget support needed to continue in proven channels. But it could be even riskier to wait too long to test or adopt some of the newer opportunities that emerge with oppressive regularity.

The bounty of options makes planning more complicated and can thinly stretch even the largest of budgets across a wide array of team efforts. Each team effort must be supported with planning, development, distribution, optimization and reporting, all of which cost time and money. And though more options generally leads to more learning, it also creates more work — and sometimes even a dilution of impact upon prospects.

Some of those new opportunities will earn key positions in future campaigns via their proven contributions to specific objectives. But many will turn out to just be a shiny object that got its fifteen minutes of marketing fame and ate away your resources. One handy tool to help you hedge this high-stakes bet is the Gartner Hype Cycle.

Gartner has been publishing this annual review for many years. It considers emerging technologies in a way that best informs critical business investments. It offers brands distinct interpretations of real value versus hype, charted along a continuum marking the highs and lows of technology adoption over time.

The cycle begins with a peak of inflated expectations, tied to a wave of adoption and a lot of market attention, before negativity and failures lead to a trough of disillusionment. Then the real work begins: adapting best practices and methodologies that lead to higher productivity. Rinse and repeat.

Hype CycleThe 2016 Gartner Hype Cycle of emerging technologies highlights three big trends, including:

  1. Immersive Consumer Experiences, like virtual reality, smart materials and gesture controls
  2. Smart Machines and workspaces that foster the evolution of the Internet of Things and digitize physical objects to improve efficiencies.
  3. Technologies that connect to each other and synergize previously autonomous technologies and platforms.

Gartner actually publishes multiple hype cycles annually. Some of these cycle reports focus on particular technologies, so if you have an interest in a specific area of technology, you should do some further digging.

It is easy to see how today’s technological innovation can evolve into tomorrow’s marketing tool kit, but it’s not a quick, direct or easy journey. Watch for the phases of the hype cycle but also for the availability of tested vendors, channels or service partners to help ease your adoption. Most marketers are not equipped to leverage the raw technology on their own, so they search for partners with a tested offering that effectively employs the emerging technology. But this typically occurs in the later stages of the hype cycle, which in some cases may be too late.

So how do you know when it’s time to jump in, and how do you maximize the impact of your inherently risky choice?

  • Have clear goals and benchmarks in place, along with a time frame to assess whether this new initiative is achieving its function within your plan.
  • Know the difference between technology and marketing. Both have value but they are not interchangeable.
  • Don’t launch what you can’t measure.
  • Some endeavors are more labor and research intensive than others, or further outside of your comfort and experience levels. Weigh the effort expended against the potential return before embarking.
  • Build in additional time. New efforts always need additional launch time, QA time, etc.
  • Fund the effort appropriately. Just dipping your toe in may not return a realistic picture of the actual value.
  • Know what your team resources can support. Unduly stressing them can have unintended negative consequences on unrelated programs that had been running smoothly prior to the adoption.
  • Keep a balance in your budget of proven tactics, but also set aside a testing budget so as to continually learn and freshen your eye.
  • Don’t hang out on the bleeding edge unless your brand and your audience are already there. Not every new marketing opportunity will be a good fit.
  • Do your research. You can learn a lot from watching early adopters.

Success today favors the bold but informed. Make smart choices, and continually test and refresh your marketing mix. Maximize the opportunity and minimize the hype.