Smart Data – Not Big Data

As a concerned data professional, I am already plotting an exit strategy from this Big Data hype. Because like any bubble, it will surely burst. That inevitable doomsday could be a couple of years away, but I can feel it coming. At the risk of sounding too much like Yoda the Jedi Grand Master, all hypes lead to over-investments, all over-investments lead to disappointments, and all disappointments lead to blames. Yes, in a few years, lots of blames will go around, and lots of heads will roll.

As a concerned data professional, I am already plotting an exit strategy from this Big Data hype. Because like any bubble, it will surely burst. That inevitable doomsday could be a couple of years away, but I can feel it coming. At the risk of sounding too much like Yoda the Jedi Grand Master, all hypes lead to over-investments, all over-investments lead to disappointments, and all disappointments lead to blames. Yes, in a few years, lots of blames will go around, and lots of heads will roll.

So, why would I stay on the troubled side? Well, because, for now, this Big Data thing is creating lots of opportunities, too. I am writing this on my way back from Seoul, Korea, where I presented this Big Data idea nine times in just two short weeks, trotting from large venues to small gatherings. Just a few years back, I used to have a hard time explaining what I do for living. Now, I just have to say “Hey, I do this Big Data thing,” and the doors start to open. In my experience, this is the best “Open Sesame” moment for all data specialists. But it will last only if we play it right.

Nonetheless, I also know that I will somehow continue to make living setting data strategies, fixing bad data, designing databases and leading analytical activities, even after the hype cools down. Just with a different title, under a different banner. I’ve seen buzzwords come and go, and this data business has been carried on by the people who cut through each hype (and gargantuan amount of BS along with it) and create real revenue-generating opportunities. At the end of the day (I apologize for using this cliché), it is all about the bottom line, whether it comes from a revenue increase or cost reduction. It is never about the buzzwords that may have created the business opportunities in the first place; it has always been more about the substance that turned those opportunities into money-making machines. And substance needs no fancy title or buzzwords attached to it.

Have you heard Google or Amazon calling themselves a “Big Data” companies? They are the ones with sick amounts of data, but they also know that it is not about the sheer amount of data, but it is all about the user experience. “Wannabes” who are not able to understand the core values often hang onto buzzwords and hypes. As if Big Data, Cloud Computing or coding language du jour will come and save the day. But they are just words.

Even the name “Big Data” is all wrong, as it implies that bigger is always better. The 3 Vs of Big Data—volume, velocity and variety—are also misleading. That could be a meaningful distinction for existing data players, but for decision-makers, it gives a notion that size and speed are the ultimate quest. But for the users, small is better. They don’t have time to analyze big sets of data. They need small answers in fun size packages. Plus, why is big and fast new? Since the invention of modern computers, has there been any year when the processing speed did not get faster and storage capacity did not get bigger?

Lest we forget, it is the software industry that came up with this Big Data thing. It was created as a marketing tagline. We should have read it as, “Yes, we can now process really large amounts of data, too,” not as, “Big Data will make all your dreams come true.” If you are in the business of selling toolsets, of course, that is how you present your product. If guitar companies keep emphasizing how hard it is to be a decent guitar player, would that help their businesses? It is a lot more effective to say, “Hey, this is the same guitar that your guitar hero plays!” But you don’t become Jeff Beck just because you bought a white Fender Stratocaster with a rosewood neck. The real hard work begins “after” you purchase a decent guitar. However, this obvious connection is often lost in the data business. Toolsets never provide solutions on their own. They may make your life easier, but you’d still have to formulate the question in a logical fashion, and still have to make decisions based on provided data. And harnessing meanings out of mounds of data requires training of your mind, much like the way musicians practice incessantly.

So, before business people even consider venturing into this Big Data hype, they should ask themselves “Why data?” What are burning questions that you are trying to solve with the data? If you can’t answer this simple question, then don’t jump into it. Forget about it. Don’t get into it just because everyone else seems to be getting into it. Yeah, it’s a big party, but why are you going there? Besides, if you formulate the question properly, often you will find that you don’t need Big Data all the time. If fact, Big Data can be a terrible detour if your question can be answered by “small” data. But that happens all the time, because people approach their business questions through the processes set by the toolsets. Big Data should be about the business, not about the IT or data.

Smart Data, Not Big Data
So, how do we get over this hype? All too often, perception rules, and a replacement word becomes necessary to summarize the essence of the concept for the general public. In my opinion, “Big Data” should have been “Smart Data.” Piles of unorganized dumb data aren’t worth a damn thing. Imagine a warehouse full of boxes with no labels, collecting dust since 1943. Would you be impressed with the sheer size of the warehouse? Great, the ark that Indiana Jones procured (or did he?) may be stored in there somewhere. But if no one knows where it is—or even if it can be located, if no one knows what to do with it—who cares?

Then, how do data get smarter? Smart data are bite-sized answers to questions. A thousand variables could have been considered to provide the weather forecast that calls for a “70 percent chance of scattered showers in the afternoon,” but that one line that we hear is the smart piece of data. Not the list of all the variables that went into the formula that created that answer. Emphasizing the raw data would be like giving paints and brushes to a person who wants a picture on the wall. As in, “Hey, here are all the ingredients, so why don’t you paint the picture and hang it on the wall?” Unfortunately, that is how the Big Data movement looks now. And too often, even the ingredients aren’t all that great.

I visit many companies only to find that the databases in question are just messy piles of unorganized and unstructured data. And please do not assume that such disarrays are good for my business. I’d rather spend my time harnessing meanings out of data and creating values, not taking care of someone else’s mess all the time. Really smart data are small, concise, clean and organized. Big Data should only be seen in “Behind the Scenes” types of documentaries for manias, not for everyday decision-makers.

I have been already saying that Big Data must get smaller for some time (refer to “Big Data Must Get Smaller“) and I would repeat it until it becomes a movement on its own. The Big Data movement must be about:

  1. Cutting down the noise
  2. Providing the answers

There is too much noise in the data, and cutting it out is the first step toward making the data smaller and smarter. The trouble is that the definition of “noise” is not static. Rock music that I grew up with was certainly a noise to my parents’ generation. In turn, some music that my kids listen to is pure noise to me. Likewise, “product color,” which is essential for a database designed for an inventory management system, may or may not be noise if the goal is to sell more apparel items. In such cases, more important variables could be style, brand, price range, target gender, etc., but color could be just peripheral information at best, or even noise (as in, “Uh, she isn’t going to buy just red shoes all the time?”). How do we then determine the differences? First, set the clear goals (as in, “Why are we playing with the data to begin with?”), define the goals using logical expressions, and let mathematics take care of it. Now you can drop the noise with conviction (even if it may look important to human minds).

If we continue with that mathematical path, we would reach the second part, which is “providing answers to the question.” And the smart answers are in the forms of yes/no, probability figures or some type of scores. Like in the weather forecast example, the question would be “chance of rain on a certain day” and the answer would be “70 percent.” Statistical modeling is not easy or simple, but it is the essential part of making the data smarter, as models are the most effective way to summarize complex and abundant data into compact forms (refer to “Why Model?”).

Most people do not have degrees in mathematics or statistics, but they all know what to do with a piece of information such as “70 percent chance of rain” on the day of a company outing. Some may complain that it is not a definite yes/no answer, but all would agree that providing information in this form is more humane than dumping all the raw data onto users. Sales folks are not necessarily mathematicians, but they would certainly appreciate scores attached to each lead, as in “more or less likely to close.” No, that is not a definite answer, but now sales people can start calling the leads in the order of relative importance to them.

So, all the Big Data players and data scientists must try to “humanize” the data, instead of bragging about the size of the data, making things more complex, and providing irrelevant pieces of raw data to users. Make things simpler, not more complex. Some may think that complexity is their job security, but I strongly disagree. That is a sure way to bring down this Big Data movement to the ground. We are already living in a complex world, and we certainly do not need more complications around us (more on “How to be a good data scientist” in a future article).

It’s About the Users, Too
On the flip side, the decision-makers must change their attitude about the data, as well.

1. Define the goals first: The main theme of this series has been that the Big Data movement is about the business, not IT or data. But I’ve seen too many business folks who would so willingly take a hands-off approach to data. They just fund the database; do not define clear business goals to developers; and hope to God that someday, somehow, some genius will show up and clear up the mess for them. Guess what? That cavalry is never coming if you are not even praying properly. If you do not know what problems you want to solve with data, don’t even get started; you will get to nowhere really slowly, bleeding lots of money and time along the way.

2. Take the data seriously: You don’t have to be a scientist to have a scientific mind. It is not ideal if someone blindly subscribes anything computers spew out (there are lots of inaccurate information in databases; refer to “Not All Databases Are Created Equal.”). But too many people do not take data seriously and continue to follow their gut feelings. Even if your customer profile coming out of a serious analysis does not match with your preconceived notions, do not blindly reject it; instead, treat it as a newly found gold mine. Gut feelings are even more overrated than Big Data.

3. Be logical: Illogical questions do not lead anywhere. There is no toolset that reads minds—at least not yet. Even if we get to have such amazing computers—as seen on “Star Trek” or in other science fiction movies—you would still have to ask questions in a logical fashion for them to be effective. I am not asking decision-makers to learn how to code (or be like Mr. Spock or his loyal follower, Dr. Sheldon Cooper), but to have some basic understanding of logical expressions and try to learn how analysts communicate with computers. This is not data geek vs. non-geek world anymore; we all have to be a little geekier. Knowing Boolean expressions may not be as cool as being able to throw a curve ball, but it is necessary to survive in the age of information overload.

4. Shoot for small successes: Start with a small proof of concept before fully investing in large data initiatives. Even with a small project, one gets to touch all necessary steps to finish the job. Understanding the flow of information is as important as each specific step, as most breakdowns occur in between steps, due to lack of proper connections. There was Gemini program before Apollo missions. Learn how to dock spaceships in space before plotting the chart to the moon. Often, over-investments are committed when the discussion is led by IT. Outsource even major components in the beginning, as the initial goal should be mastering the flow of things.

5. Be buyer-centric: No customer is bound by the channel of the marketer’s choice, and yet, may businesses act exactly that way. No one is an online person just because she did not refuse your email promotions yet (refer to “The Future of Online is Offline“). No buyer is just one dimensional. So get out of brand-, division-, product- or channel-centric mindsets. Even well-designed, buyer-centric marketing databases become ineffective if users are trapped in their channel- or division-centric attitudes, as in “These email promotions must flow!” or “I own this product line!” The more data we collect, the more chances marketers will gain to impress their customers and prospects. Do not waste those opportunities by imposing your own myopic views on them. Big Data movement is not there to fortify marketers’ bad habits. Thanks to the size of the data and speed of machines, we are now capable of disappointing a lot of people really fast.

What Did This Hype Change?
So, what did this Big Data hype change? First off, it changed people’s attitudes about the data. Some are no longer afraid of large amounts of information being thrown at them, and some actually started using them in their decision-making processes. Many realized that we are surrounded by numbers everywhere, not just in marketing, but also in politics, media, national security, health care and the criminal justice system.

Conversely, some people became more afraid—often with good reasons. But even more often, people react based on pure fear that their personal information is being actively exploited without their consent. While data geeks are rejoicing in the age of open source and cloud computing, many more are looking at this hype with deep suspicions, and they boldly reject storing any personal data in those obscure “clouds.” There are some people who don’t even sign up for EZ Pass and voluntarily stay on the long lane to pay tolls in the old, but untraceable way.

Nevertheless, not all is lost in this hype. The data got really big, and types of data that were previously unavailable, such as mobile and social data, became available to many marketers. Focus groups are now the size of Twitter followers of the company or a subject matter. The collection rate of POS (point of service) data has been increasingly steady, and some data players became virtuosi in using such fresh and abundant data to impress their customers (though some crossed that “creepy” line inadvertently). Different types of data are being used together now, and such merging activities will compound the predictive power even further. Analysts are dealing with less missing data, though no dataset would ever be totally complete. Developers in open source environments are now able to move really fast with new toolsets that would just run on any device. Simply, things that our forefathers of direct marketing used to take six months to complete can be done in few hours, and in the near future, maybe within a few seconds.

And that may be a good thing and a bad thing. If we do this right, without creating too many angry consumers and without burning holes in our budgets, we are currently in a position to achieve great many things in terms of predicting the future and making everyone’s lives a little more convenient. If we screw it up badly, we will end up creating lots of angry customers by abusing sensitive data and, at the same time, wasting a whole lot of investors’ money. Then this Big Data thing will go down in history as a great money-eating hype.

We should never do things just because we can; data is a powerful tool that can hurt real people. Do not even get into it if you don’t have a clear goal in terms of what to do with the data; it is not some piece of furniture that you buy just because your neighbor bought it. Living with data is a lifestyle change, and it requires a long-term commitment; it is not some fad that you try once and give up. It is a continuous loop where people’s responses to marketer’s data-based activities create even more data to be analyzed. And that is the only way it keeps getting better.

There Is No Big Data
And all that has nothing to do with “Big.” If done right, small data can do plenty. And in fact, most companies’ transaction data for the past few years would easily fit in an iPhone. It is about what to do with the data, and that goal must be set from a business point of view. This is not just a new playground for data geeks, who may care more for new hip technologies that sound cool in their little circle.

I recently went to Brazil to speak at a data conference called QIBRAS, and I was pleasantly surprised that the main theme of it was the quality of the data, not the size of the data. Well, at least somewhere in the world, people are approaching this whole thing without the “Big” hype. And if you look around, you will not find any successful data players calling this thing “Big Data.” They just deal with small and large data as part of their businesses. There is no buzzword, fanfare or a big banner there. Because when something is just part of your everyday business, you don’t even care what you call it. You just do. And to those masters of data, there is no Big Data. If Google all of a sudden starts calling itself a Big Data company, it would be so uncool, as that word would seriously limit it. Think about that.

The Best Brand Gift Ever!

I know you are a YES person. A DIY person. A BRING IT person. A CAN DO person … excellent at all you do—conscientious, responsible, dependable, overachieving. No doubt, it’s how you got where you are. All wonderful qualities. So this Christmas, perhaps the last thing you need or want is something from “The 12 Days of Christmas.” What you just might need this month is 12 days and ways to say NO.

I know you are a YES person. A DIY person. A BRING IT person. A CAN DO person … excellent at all you do—conscientious, responsible, dependable, overachieving. No doubt, it’s how you got where you are. All wonderful qualities. So this Christmas, perhaps the last thing you need or want is, as the song says, some version of “12 drummers drumming, 11 pipers piping, 10 lords-a-leaping, nine ladies dancing, eight maids-a-milking, seven swans-a-swimming, six geese-a-laying, five golden rings, four calling birds, three French hens, two turtle doves or even a partridge in a pear tree.” You don’t need or want more stuff. You want a meaningful, long-lasting, brand-enhancing and life-affirming gift. Something useful and practical.

What you just might need this month is 12 days and ways to say NO.

The deal is that no one can give this gift to you. It’s a selfie. There’s no outsourcing this skill to a personal shopper, no concierge service that can do this for you. It’s a true DIYer.

As YES people, the word NO is an infrequent part of our vocabulary—in our brand lives and in our personal lives. But I have found that the happiest and most productive people have given themselves the gift of NO. They have learned to make NO a natural part of their DNA … both in and out of the office.

So, before you head out of the office to start holiday celebrations, why not raise a toast to that little two-letter word NO and see if these bits of inspiration may encourage you to treat yourself (and the brand you lead) to this very important present:

1. The gift of a new discipline … making no an art form. Missy Park, founder of Title Nine, echoes the power of no. “In my book, saying yes is over-rated. Fact is, it’s easy to say yes. No difficult choices, no disappointments. Ahh, but saying no. That is the real art form. There’s choosing to say no which can be wrenching. There is choosing when to say no, which is often. And then there’s saying it graciously, which is very hard indeed.”

2. The gift of throwing in the towel … the towel that really doesn’t matter. I greatly admire Bob Goff. He’s an author, an attorney and founder of Restore International, a nonprofit human rights organization. He wisely shares: “I used to be afraid of failing at something that really mattered to me, but now I’m more afraid of succeeding at things that don’t matter.” With that in mind, Goff makes it a habit to quit something every Thursday. It liberates him for new things. What can you be simply done with?

3. The gift of margin … build in white space … everywhere! Dr. Richard Swensen, a physician-futurist, educator and author, advocates for purposefully creating mental, emotional, physical and spiritual breathing room in our full-to-brimming professional and personal lives. He calls it margin—like the white space around pages of books. He counsels that we need it more than ever. Appropriately saying NO gives us more white space.

4. The gift of focus … just say no … perhaps three times or more! Steve Jobs, Apple’s brilliant and passionate founder, shared this: “Focusing is about saying no. You’ve got say no, no, no. The result of that focus is going to be some really great products where the total is much greater than the sum of the parts.”

5. The gift of eliminating even more non-urgent and unimportant time fritters. Stephen Covey, author of “Seven Habits of Highly Effective People” cautions us to be careful of defaulting too often into what he calls Quadrant 4 of his time management matrix … the place we naturally drift after spending lots of time in urgent and crisis modes: trivia, busywork, mindless surfing. Just say goodbye to all the nonessentials.

6. The gift of stopping … count the ways. Jim Collins, author of “Good to Great,” encourages us to create STOP DOING LISTS. That’s right … enumerate all things you are no longer going to do. Start by simply saying no to his Venn diagram of three crucial things-activities that are you are not deeply passionate about, that you feel you are not genetically encoded for and things that don’t make much economic sense.

7. The gift of holding back … a permission slip for more B+s. Must everything be done to an A+ perfection level? Pick and choose those activities that really warrant this kind of energy. Challenge yourself to not be an honors student in all you do. Award-winning author Anne Lamott had to remind herself in midlife that “a B+ is just fine.”

8. The gift of less … hit that delete key more often. Do we really need (or have time to read) all those subscriptions? Must we? Find satisfaction in architect Ludwig Mies van der Rohe “less is more” philosophy. Go ahead—delete, unsubscribe, edit, curate. Whatever you have to call this process, just do it.

9. The gift of simplicity … now. Years ago naturalist and poet, Henry David Thoreau warned us: “Our life is frittered away by detail … Simplify, simplify, simplify!” Alan Seigel updates that sentiment for brand leaders in his book: Simple: Conquering the Crisis of Complexity. Perhaps it’s time to give yourself and your brand the gift of a serious simplification process.

10. The gift of benign neglect … just ignore it! Do we really have to have a multiplatform constantly clean inbox? Who cares? What’s the point? Mani S. Sivasubramanian, author of “How To Focus – Stop Procrastinating, Improve Your Concentration & Get Things Done – Easily!” writes: “Information overload (on all levels) is exactly WHY you need an “ignore list.” It has never been more important to be able to say “No.”

11. The gift of checking back in with yourself … so, what matters now? In her book “Fierce Conversations,” leadership development architect Susan Scott suggests people change and forget to tell one another. That is true. Sometimes we even forget to tell ourselves. What has changed for you or your brand? Your energy level? Your tolerance? Your interests? Your competition? Your customers? What needs revisiting so that your yeses are truly yeses and your nos are truly nos?

12. The gift of a do-over … recycle your mistakes. We’ve all made the mistake of saying yes when we should have said no. Jot down a few of those do-overs on a post it note. What were the learning lessons? Keep that note to yourself handy.

‘Tis the season for gift-giving. Be kind to yourself and to your brand and make the practice of gracious NO saying not only a year end gift, but a long lasting part of your DNA.

5 Tips for Faster, More Confident, Direct Marketing Budget Decisions

As we enter the critical make-or-break fourth quarter, and you begin your 2014 direct marketing budget plans, you will likely be faced with many marketing decisions. Those decisions are usually needed quickly. But often they’re not made quickly. Whether it’s information overload from so many options, analysis paralysis or managers who are afraid to make a decision, today we explore five ways to

As we enter the critical make-or-break fourth quarter, and you begin your 2014 direct marketing budget plans, you will likely be faced with many marketing decisions. Those decisions are usually needed quickly. But often they’re not made quickly. Whether it’s information overload from so many options, analysis paralysis or managers who are afraid to make a decision, today we explore five ways to make marketing decisions quicker and more confidently.

A mere generation ago, direct marketing decisions were limited to direct mail customer file or rented lists, space ads in magazines, package inserts, direct response broadcast, and a few other media options.

Fast forward to now, and the direct marketing decision landscape has grown exponentially with online and cross-promotional media options. Every season reveals new, unexplored online opportunities. Some are fads. Some turn out to have real value.

So for your direct marketing budget planning, here are five recommendations of how to evaluate opportunities and make decisions more quickly and confidently.

1. Cost per Response
An important metric for most direct marketers is the marketing cost per response (per lead, inquiry, sale—whatever your situation). This core metric may be your most significant contributor to your decisions.

2. Allocation of Unknown Response Sources
If you’re in a situation where you have a significant number of responses for which you can’t pinpoint a specific marketing source, consider a weighted-average allocation of those responses across marketing activities. With some imagination, you should be able to calculate this on your own. (Let me know if you’d like an expansion of this concept, and perhaps we’ll do so in a future blog post.)

3. Summarize Results in a Matrix
Placing your data in a spreadsheet will put the numbers in front of you so you can see all your activity in one place. You may want the data by media type on separate spreadsheet tabs so you can see more granular data.

For example, on one tab you summarize results from direct mail (by list, or summed up by customer vs. rented lists) with cost per response. If you allocated unknown orders, be sure to include those. Another tab might concern email results that summarize opens, clicks, conversions and cost per response. Other tabs could summarize pay-per-click, social media, retargeting or whatever media you are using. Then roll up and summarize all of the media on a tab of its own. If cost per response is most important to you, then sort the data from the lowest cost per response to highest. Perhaps you have “soft data” that will be a factor in your decisions. If so, add columns to enable a written evaluation of each. Maybe your evaluation is as simple as “pluses” and “minuses” for each opportunity.

4. Parameters for Decisions
It happens all the time. With so many choices and options, and potentially several staff members wanting their piece of the budget, decisions can be contentious and slow. When that happens, everyone loses. When you establish the parameters for decision making upfront, it’s easier to slice the pie into the right proportions. More importantly, if the head of the organization or department has established those parameters in writing (avoid verbal direction to avoid future misunderstanding), staff is empowered to make more confident decisions without delay.

5. Don’t Forget Test Budgets
Know, ahead of time, how much money you can gamble in a test. You should view the money spent as having zero return so that when if it works you’re pleasantly surprised. A rule of thumb you might use is to allocate 10 percent of a total marketing budget to tests. Whether it’s a direct mail list test, or new online media, the only way you can learn if those options work for you is to test it. Remember, too, that marketing fads can fizzle quickly. The hot new opportunity of 2012—not even a full year ago—may already be a distant memory.

If you have processes, or recommendations, about how you make faster, more confident marketing decisions, please share them in the comments area below.