Missing Data Can Be Meaningful

No matter how big the Big Data gets, we will never know everything about everything. Well, according to the super-duper computer called “Deep Thought” in the movie “The Hitchhiker’s Guide to the Galaxy” (don’t bother to watch it if you don’t care for the British sense of humour), the answer to “The Ultimate Question of Life, the Universe, and Everything” is “42.” Coincidentally, that is also my favorite number to bet on (I have my reasons), but I highly doubt that even that huge fictitious computer with unlimited access to “everything” provided that numeric answer with conviction after 7½ million years of computing and checking. At best, that “42” is an estimated figure of a sort, based on some fancy algorithm. And in the movie, even Deep Thought pointed out that “the answer is meaningless, because the beings who instructed it never actually knew what the Question was.” Ha! Isn’t that what I have been saying all along? For any type of analytics to be meaningful, one must properly define the question first. And what to do with the answer that comes out of an algorithm is entirely up to us humans, or in the business world, the decision-makers. (Who are probably human.)

No matter how big the Big Data gets, we will never know everything about everything. Well, according to the super-duper computer called “Deep Thought” in the movie “The Hitchhiker’s Guide to the Galaxy” (don’t bother to watch it if you don’t care for the British sense of humour), the answer to “The Ultimate Question of Life, the Universe, and Everything” is “42.” Coincidentally, that is also my favorite number to bet on (I have my reasons), but I highly doubt that even that huge fictitious computer with unlimited access to “everything” provided that numeric answer with conviction after 7½ million years of computing and checking. At best, that “42” is an estimated figure of a sort, based on some fancy algorithm. And in the movie, even Deep Thought pointed out that “the answer is meaningless, because the beings who instructed it never actually knew what the Question was.” Ha! Isn’t that what I have been saying all along? For any type of analytics to be meaningful, one must properly define the question first. And what to do with the answer that comes out of an algorithm is entirely up to us humans, or in the business world, the decision-makers. (Who are probably human.)

Analytics is about making the best of what we know. Good analysts do not wait for a perfect dataset (it will never come by, anyway). And businesspeople have no patience to wait for anything. Big Data is big because we digitize everything, and everything that is digitized is stored somewhere in forms of data. For example, even if we collect mobile device usage data from just pockets of the population with certain brands of mobile services in a particular area, the sheer size of the resultant dataset becomes really big, really fast. And most unstructured databases are designed to collect and store what is known. If you flip that around to see if you know every little behavior through mobile devices for “everyone,” you will be shocked to see how small the size of the population associated with meaningful data really is. Let’s imagine that we can describe human beings with 1,000 variables coming from all sorts of sources, out of 200 million people. How many would have even 10 percent of the 1,000 variables filled with some useful information? Not many, and definitely not 100 percent. Well, we have more data than ever in the history of mankind, but still not for every case for everyone.

In my previous columns, I pointed out that decision-making is about ranking different options, and to rank anything properly. We must employee predictive analytics (refer to “It’s All About Ranking“). And for ranking based on the scores resulting from predictive models to be effective, the datasets must be summarized to the level that is to be ranked (e.g., individuals, households, companies, emails, etc.). That is why transaction or event-level datasets must be transformed to “buyer-centric” portraits before any modeling activity begins. Again, it is not about the transaction or the products, but it is about the buyers, if you are doing all this to do business with people.

Trouble with buyer- or individual-centric databases is that such transformation of data structure creates lots of holes. Even if you have meticulously collected every transaction record that matters (and that will be the day), if someone did not buy a certain item, any variable that is created based on the purchase record of that particular item will have nothing to report for that person. Likewise, if you have a whole series of variables to differentiate online and offline channel behaviors, what would the online portion contain if the consumer in question never bought anything through the Web? Absolutely nothing. But in the business of predictive analytics, what did not happen is as important as what happened. Even a simple concept of “response” is only meaningful when compared to “non-response,” and the difference between the two groups becomes the basis for the “response” model algorithm.

Capturing the Meanings Behind Missing Data
Missing data are all around us. And there are many reasons why they are missing, too. It could be that there is nothing to report, as in aforementioned examples. Or, there could be errors in data collection—and there are lots of those, too. Maybe you don’t have access to certain pockets of data due to corporate, legal, confidentiality or privacy reasons. Or, maybe records did not match properly when you tried to merge disparate datasets or append external data. These things happen all the time. And, in fact, I have never seen any dataset without a missing value since I left school (and that was a long time ago). In school, the professors just made up fictitious datasets to emphasize certain phenomena as examples. In real life, databases have more holes than Swiss cheese. In marketing databases? Forget about it. We all make do with what we know, even in this day and age.

Then, let’s ask a philosophical question here:

  • If missing data are inevitable, what do we do about it?
  • How would we record them in databases?
  • Should we just leave them alone?
  • Or should we try to fill in the gaps?
  • If so, how?

The answer to all this is definitely not 42, but I’ll tell you this: Even missing data have meanings, and not all missing data are created equal, either.

Furthermore, missing data often contain interesting stories behind them. For example, certain demographic variables may be missing only for extremely wealthy people and very poor people, as their residency data are generally not exposed (for different reasons, of course). And that, in itself, is a story. Likewise, some data may be missing in certain geographic areas or for certain age groups. Collection of certain types of data may be illegal in some states. “Not” having any data on online shopping behavior or mobile activity may mean something interesting for your business, if we dig deeper into it without falling into the trap of predicting legal or corporate boundaries, instead of predicting consumer behaviors.

In terms of how to deal with missing data, let’s start with numeric data, such as dollars, days, counters, etc. Some numeric data simply may not be there, if there is no associated transaction to report. Now, if they are about “total dollar spending” and “number of transactions” in a certain category, for example, they can be initiated as zero and remain as zero in cases like this. The counter simply did not start clicking, and it can be reported as zero if nothing happened.

Some numbers are incalculable, though. If you are calculating “Average Amount per Online Transaction,” and if there is no online transaction for a particular customer, that is a situation for mathematical singularity—as we can’t divide anything by zero. In such cases, the average amount should be recorded as: “.”, blank, or any value that represents a pure missing value. But it should never be recorded as zero. And that is the key in dealing with missing numeric information; that zero should be reserved for real zeros, and nothing else.

I have seen too many cases where missing numeric values are filled with zeros, and I must say that such a practice is definitely frowned-upon. If you have to pick just one takeaway from this article, that’s it. Like I emphasized, not all missing values are the same, and zero is not the way you record them. Zeros should never represent lack of information.

Take the example of a popular demographic variable, “Number of Children in the Household.” This is a very predictable variable—not just for purchase behavior of children’s products, but for many other things. Now, it is a simple number, but it should never be treated as a simple variable—as, in this case, lack of information is not the evidence of non-existence. Let’s say that you are purchasing this data from a third-party data compiler (or a data broker). If you don’t see a positive number in that field, it could be because:

  1. The household in question really does not have a child;
  2. Even the data-collector doesn’t have the information; or
  3. The data collector has the information, but the household record did not match to the vendor’s record, for some reason.

If that field contains a number like 1, 2 or 3, that’s easy, as they will represent the number of children in that household. But the zero should be reserved for cases where the data collector has a positive confirmation that the household in question indeed does not have any children. If it is unknown, it should be marked as blank, “.” (Many statistical softwares, such as SAS, record missing values this way.) Or use “U” (though an alpha character should not be in a numeric field).

If it is a case of non-match to the external data source, then there should be a separate indicator for it. The fact that the record did not match to a professional data compiler’s list may mean something. And I’ve seen cases where such non-matching indicators are made to model algorithms along with other valid data, as in the case where missing indicators of income display the same directional tendency as high-income households.

Now, if the data compiler in question boldly inputs zeros for the cases of unknowns? Take a deep breath, fire the vendor, and don’t deal with the company again, as it is a sign that its representatives do not know what they are doing in the data business. I have done so in the past, and you can do it, too. (More on how to shop for external data in future articles.)

For non-numeric categorical data, similar rules apply. Some values could be truly “blank,” and those should be treated separately from “Unknown,” or “Not Available.” As a practice, let’s list all kinds of possible missing values in codes, texts or other character fields:

  • ” “—blank or “null”
  • “N/A,” “Not Available,” or “Not Applicable”
  • “Unknown”
  • “Other”—If it is originating from some type of multiple choice survey or pull-down menu
  • “Not Answered” or “Not Provided”—This indicates that the subjects were asked, but they refused to answer. Very different from “Unknown.”
  • “0”—In this case, the answer can be expressed in numbers. Again, only for known zeros.
  • “Non-match”—Not matched to other internal or external data sources
  • Etc.

It is entirely possible that all these values may be highly correlated to each other and move along the same predictive direction. However, there are many cases where they do not. And if they are combined into just one value, such as zero or blank, we will never be able to detect such nuances. In fact, I’ve seen many cases where one or more of these missing indicators move together with other “known” values in models. Again, missing data have meanings, too.

Filling in the Gaps
Nonetheless, missing data do not have to left as missing, blank or unknown all the time. With statistical modeling techniques, we can fill in the gaps with projected values. You didn’t think that all those data compilers really knew the income level of every household in the country, did you? It is not a big secret that much of those figures are modeled with other available data.

Such inferred statistics are everywhere. Popular variables, such as householder age, home owner/renter indicator, housing value, household income or—in the case of business data—the number of employees and sales volume contain modeled values. And there is nothing wrong with that, in the world where no one really knows everything about everything. If you understand the limitations of modeling techniques, it is quite alright to employ modeled values—which are much better alternatives to highly educated guesses—in decision-making processes. We just need to be a little careful, as models often fail to predict extreme values, such as household incomes over $500,000/year, or specific figures, such as incomes of $87,500. But “ranges” of household income, for example, can be predicted at a high confidence level, though it technically requires many separate algorithms and carefully constructed input variables in various phases. But such technicality is an issue that professional number crunchers should deal with, like in any other predictive businesses. Decision-makers should just be aware of the reality of real and inferred data.

Such imputation practices can be applied to any data source, not just compiled databases by professional data brokers. Statisticians often impute values when they encounter missing values, and there are many different methods of imputation. I haven’t met two statisticians who completely agree with each other when it comes to imputation methodologies, though. That is why it is important for an organization to have a unified rule for each variable regarding its imputation method (or lack thereof). When multiple analysts employ different methods, it often becomes the very source of inconsistent or erroneous results at the application stage. It is always more prudent to have the calculation done upfront, and store the inferred values in a consistent manner in the main database.

In terms of how that is done, there could be a long debate among the mathematical geeks. Will it be a simple average of non-missing values? If such a method is to be employed, what is the minimum required fill-rate of the variable in question? Surely, you do not want to project 95 percent of the population with 5 percent known values? Or will the missing values be replaced with modeled values, as in previous examples? If so, what would be the source of target data? What about potential biases that may exist because of data collection practices and their limitations? What should be the target definition? In what kind of ranges? Or should the target definition remain as a continuous figure? How would you differentiate modeled and real values in the database? Would you embed indicators for inferred values? Or would you forego such flags in the name of speed and convenience for users?

The important matter is not the rules or methodologies, but the consistency of them throughout the organization and the databases. That way, all users and analysts will have the same starting point, no matter what the analytical purposes are. There could be a long debate in terms of what methodology should be employed and deployed. But once the dust settles, all data fields should be treated by pre-determined rules during the database update processes, avoiding costly errors in the downstream. All too often, inconsistent imputation methods lead to inconsistent results.

If, by some chance, individual statisticians end up with freedom to come up with their own ways to fill in the blanks, then the model-scoring code in question must include missing value imputation algorithms without an exception, granted that such practice will elongate the model application processes and significantly increase chances for errors. It is also important that non-statistical users should be educated about the basics of missing data and associated imputation methods, so that everyone who has access to the database shares a common understanding of what they are dealing with. That list includes external data providers and partners, and it is strongly recommended that data dictionaries must include employed imputation rules wherever applicable.

Keep an Eye on the Missing Rate
Often, we get to find out that the missing rate of certain variables is going out of control because models become ineffective and campaigns start to yield disappointing results. Conversely, it can be stated that fluctuations in missing data ratios greatly affect the predictive power of models or any related statistical works. It goes without saying that a consistent influx of fresh data matters more than the construction and the quality of models and algorithms. It is a classic case of a garbage-in-garbage-out scenario, and that is why good data governance practices must include a time-series comparison of the missing rate of every critical variable in the database. If, all of a sudden, an important predictor’s fill-rate drops below a certain point, no analyst in this world can sustain the predictive power of the model algorithm, unless it is rebuilt with a whole new set of variables. The shelf life of models is definitely finite, but nothing deteriorates effectiveness of models faster than inconsistent data. And a fluctuating missing rate is a good indicator of such an inconsistency.

Likewise, if the model score distribution starts to deviate from the original model curve from the development and validation samples, it is prudent to check the missing rate of every variable used in the model. Any sudden changes in model score distribution are a good indicator that something undesirable is going on in the database (more on model quality control in future columns).

These few guidelines regarding the treatment of missing data will add more flavors to statistical models and analytics in general. In turn, proper handling of missing data will prolong the predictive power of models, as well. Missing data have hidden meanings, but they are revealed only when they are treated properly. And we need to do that until the day we get to know everything about everything. Unless you are just happy with that answer of “42.”

1-Trick Ponies and Customer Loyalty Behavior

About 30 years ago, Paul Simon wrote a song entitled “One-Trick Pony.” The song describes a performing pony that has learned only one trick, and he succeeds or fails with the audience based on how well he executes it. As Simon conveys in the lyrics: “He’s got one trick to last a lifetime. It’s the principal source of his revenue.”

About 30 years ago, Paul Simon wrote a song entitled “One-Trick Pony.” The song describes a performing pony that has learned only one trick, and he succeeds or fails with the audience based on how well he executes it. As Simon conveys in the lyrics: “He’s got one trick to last a lifetime. It’s the principal source of his revenue.”

For a long time, I’ve seen this song and its message as something of a metaphor for what challenges many companies endeavoring to create customer loyalty behavior and more effective customer loyalty programs.

A key reason companies have a difficult time achieving stronger customer loyalty is they fail to provide full value and emotional relationship fundamentals. They focus on satisfying customers exclusively through basic rational and functional benefits, which is often too benign and passive an approach to create lasting value.

Mostly, they emphasize single-element or minimal element tactical approaches with customers, such as pricing, merchandise, loyalty cards or points-based programs, without determining (either before programs are launched or after they are up and running) whether this is sufficient motivation for building a long-term relationship. Smart marketers know, for instance, being a low-cost provider can be a trap and that only overall perceived value will prevail. In the United States, chain discount retailers like Caldor, Bradlees, Jamesway, Value City, Ames and Filene’s are either in trouble or have gone out of business, while Target, Costco and Walmart, with strong brand equity and high perceived value, have sustained.

Being a low-cost provider means that brand and customer strategies get little emphasis, and they require little investment. Let’s be honest. Cutting costs seems safe. The downside is it usually does not produce much loyalty (customer or staff), strategic differentiation or profitability.

In a 1980 Harvard Business Review article by William Hall (written, parenthetically, about the same time Simon wrote “One-Trick Pony), he reported study results comparing companies that competed on differentiated customer value vs. companies that competed principally on cost. On any important measure—return on equity, return on capital, and annual revenue growth—companies delivering both rational and relationship value beat the price competitors every time.

Customers can almost always locate cheaper products or services. Ultimately, they will invest a greater share of their purchase dollars with suppliers who create stronger emotional bonds and deliver superior perceived value. Competing on price, or any other single dimension, may pull away customers from other suppliers in the short run, but it will be difficult to keep them for long. Price is rarely a sufficient “barrier to exit,” and is more often an invitation to churn.

The same thing often holds true for incentive programs. Many consumers participate in programs like supermarket bonus clubs and airline frequent flyer programs, but they aren’t particularly effective at producing greater loyalty for any one airline or any one supermarket chain. Customers are often members of several programs, and the most active users tend to be those who would have been frequent purchasers, anyway. The incentive and reward structure more often benefits the already loyal rather than increasing loyalty. Gift programs, travel, dining, entertainment, merchandise, and cash award programs, and other plateau and pre-selected response stimulus programs are having an increasingly difficult time breaking through the clutter to provide unique, differentiated customer value.

Some of the online incentive programs have positively increased transactions, mostly among younger, female and active surfing potential buyers. To keep these incentive promotions from being one-trick ponies, they must be carefully targeted to the right consumers and at the right time. These programs must have four effective elements: ability to attract prospects to the website and, once there, to generate consideration, preference, and purchase. Getting infrequent buyers to purchase more often, or frequent buyers to place larger orders through the use of incentives, will hinge on how well companies leverage their customers’ profiles. Even more basic, it must be well-understood what customers perceive as value and what it will take to optimize their repeat purchases. The essentials for bricks and mortar product and service providers are virtually the same.

Generic, cookie-cutter and “me, too” discounts or incentives don’t do particularly well at increasing overall customer “share of wallet,” because they don’t sufficiently reward the customer for their enhanced purchase activity over time. All that’s really required to meet the customer halfway is infusion of some targeted, personalized elements to the incentive program to make them more attractive and beneficial.

The first step is to segment customers who should receive different incentives. This can be done through both basic data analysis and applied, or pilot, customer research. For example, for large customers who purchase infrequently, the company might have determined that, if they offer special discounts made within the near future, say 60 or 90 days, these customers would find that attractive. Customers who purchase frequently but in low volume amounts might be offered a discount on their next order, so long as it is larger than their last order. The array of potential loyalty program offerings can be customized based on identified needs.

What about incentives for customers who are both frequent and large volume purchasers? Well, start by saying “thank you” to them. Few things are more appreciated than thanks, and few companies express their gratitude as much as they should. Many forget to thank their customers altogether. This is especially critical for Web-based companies, or ISPs and cable companies, where the purchase experience is frequently virtual rather than personal. Thanked customers are more likely to go out of their way to provide positive referrals and testimonials.

Paul Simon’s song lyrics conclude: ” … the bag of tricks it takes to get me through my working day.” Companies would be well-served to have a bag of experience and customer loyalty tricks, using disciplined research and customer data to identify them, rather than relying on only one—price—to get them through.

Big Data Must Get Smaller

Like many folks who worked in the data business for a long time, I don’t even like the words “Big Data.” Yeah, data is big now, I get it. But so what? Faster and bigger have been the theme in the computing business since the first calculator was invented. In fact, I don’t appreciate the common definition of Big Data that is often expressed in the three Vs: volume, velocity and variety. So, if any kind of data are big and fast, it’s all good? I don’t think so. If you have lots of “dumb” data all over the place, how does that help you? Well, as much as all the clutter that’s been piled on in your basement since 1971. It may yield some profit on an online auction site one day. Who knows? Maybe some collector will pay good money for some obscure Coltrane or Moody Blues albums that you never even touched since your last turntable (Ooh, what is that?) died on you. Those oversized album jackets were really cool though, weren’t they?

Like many folks who worked in the data business for a long time, I don’t even like the words “Big Data.” Yeah, data is big now, I get it. But so what? Faster and bigger have been the theme in the computing business since the first calculator was invented. In fact, I don’t appreciate the common definition of Big Data that is often expressed in the three Vs: volume, velocity and variety. So, if any kind of data are big and fast, it’s all good? I don’t think so. If you have lots of “dumb” data all over the place, how does that help you? Well, as much as all the clutter that’s been piled on in your basement since 1971. It may yield some profit on an online auction site one day. Who knows? Maybe some collector will pay good money for some obscure Coltrane or Moody Blues albums that you never even touched since your last turntable (Ooh, what is that?) died on you. Those oversized album jackets were really cool though, weren’t they?

Seriously, the word “Big” only emphasizes the size element, and that is a sure way to miss the essence of the data business. And many folks are missing even that little point by calling all decision-making activities that involve even small-sized data “Big Data.” It is entirely possible that this data stuff seems all new to someone, but the data-based decision-making process has been with us for a very long time. If you use that “B” word to differentiate old-fashioned data analytics of yesteryear and ridiculously large datasets of the present day, yes, that is a proper usage of it. But we all know most people do not mean it that way. One side benefit of this bloated and hyped up buzzword is data professionals like myself do not have to explain what we do for living for 20 minutes anymore by simply uttering the word “Big Data,” though that is a lot like a grandmother declaring all her grandchildren work on computers for living. Better yet, that magic “B” word sometimes opens doors to new business opportunities (or at least a chance to grab a microphone in non-data-related meetings and conferences) that data geeks of the past never dreamed of.

So, I guess it is not all that bad. But lest we forget, all hypes lead to overinvestments, and all overinvestments leads to disappointments, and all disappointments lead to purging of related personnel and vendors that bear that hyped-up dirty word in their titles or division names. If this Big Data stuff does not yield significant profit (or reduction in cost), I am certain that those investment bubbles will burst soon enough. Yes, some data folks may be lucky enough to milk it for another two or three years, but brace for impact if all those collected data do not lead to some serious dollar signs. I know how the storage and processing cost decreased significantly in recent years, but they ain’t totally free, and related man-hours aren’t exactly cheap, either. Also, if this whole data business is a new concept to an organization, any money spent on the promise of Big Data easily becomes a liability for the reluctant bunch.

This is why I open up my speeches and lectures with this question: “Have you made any money with this Big Data stuff yet?” Surely, you didn’t spend all that money to provide faster toys and nicer playgrounds to IT folks? Maybe the head of IT had some fun with it, but let’s ask that question to CFOs, not CTOs, CIOs or CDOs. I know some colleagues (i.e., fellow data geeks) who are already thinking about a new name for this—”decision-making activities, based on data and analytics”—because many of us will be still doing that “data stuff” even after Big Data cease to be cool after the judgment day. Yeah, that Gangnam Style dance was fun for a while, but who still jumps around like a horse?

Now, if you ask me (though nobody did yet), I’d say the Big Data should have been “Smart Data,” “Intelligent Data” or something to that extent. Because data must provide insights. Answers to questions. Guidance to decision-makers. To data professionals, piles of data—especially the ones that are fragmented, unstructured and unformatted, no matter what kind of fancy names the operating system and underlying database technology may bear—it is just a good start. For non-data-professionals, unrefined data—whether they are big or small—would remain distant and obscure. Offering mounds of raw data to end-users is like providing a painting kit when someone wants a picture on the wall. Bragging about the size of the data with impressive sounding new measurements that end with “bytes” is like counting grains of rice in California in front of a hungry man.

Big Data must get smaller. People want yes/no answers to their specific questions. If such clarity is not possible, probability figures to such questions should be provided; as in, “There’s an 80 percent chance of thunderstorms on the day of the company golf outing,” “An above-average chance to close a deal with a certain prospect” or “Potential value of a customer who is repeatedly complaining about something on the phone.” It is about easy-to-understand answers to business questions, not a quintillion bytes of data stored in some obscure cloud somewhere. As I stated at the end of my last column, the Big Data movement should be about (1) Getting rid of the noise, and (2) Providing simple answers to decision-makers. And getting to such answers is indeed the process of making data smaller and smaller.

In my past columns, I talked about the benefits of statistical models in the age of Big Data, as they are the best way to compact big and complex information in forms of simple answers (refer to “Why Model?”). Models built to predict (or point out) who is more likely to be into outdoor sports, to be a risk-averse investor, to go on a cruise vacation, to be a member of discount club, to buy children’s products, to be a bigtime donor or to be a NASCAR fan, are all providing specific answers to specific questions, while each model score is a result of serious reduction of information, often compressing thousands of variables into one answer. That simplification process in itself provides incredible value to decision-makers, as most wouldn’t know where to cut out unnecessary information to answer specific questions. Using mathematical techniques, we can cut down the noise with conviction.

In model development, “Variable Reduction” is the first major step after the target variable is determined (refer to “The Art of Targeting“). It is often the most rigorous and laborious exercise in the whole model development process, where the characteristics of models are often determined as each statistician has his or her unique approach to it. Now, I am not about to initiate a debate about the best statistical method for variable reduction (I haven’t met two statisticians who completely agree with each other in terms of methodologies), but I happened to know that many effective statistical analysts separate variables in terms of data types and treat them differently. In other words, not all data variables are created equal. So, what are the major types of data that database designers and decision-makers (i.e., non-mathematical types) should be aware of?

In the business of predictive analytics for marketing, the following three types of data make up three dimensions of a target individual’s portrait:

  1. Descriptive Data
  2. Transaction Data / Behavioral Data
  3. Attitudinal Data

In other words, if we get to know all three aspects of a person, it will be much easier to predict what the person is about and/or what the person will do. Why do we need these three dimensions? If an individual has a high income and is living in a highly valued home (demographic element, which is descriptive); and if he is an avid golfer (behavioral element often derived from his purchase history), can we just assume that he is politically conservative (attitudinal element)? Well, not really, and not all the time. Sometimes we have to stop and ask what the person’s attitude and outlook on life is all about. Now, because it is not practical to ask everyone in the country about every subject, we often build models to predict the attitudinal aspect with available data. If you got a phone call from a political party that “assumes” your political stance, that incident was probably not random or accidental. Like I emphasized many times, analytics is about making the best of what is available, as there is no such thing as a complete dataset, even in this age of ubiquitous data. Nonetheless, these three dimensions of the data spectrum occupy a unique and distinct place in the business of predictive analytics.

So, in the interest of obtaining, maintaining and utilizing all possible types of data—or, conversely, reducing the size of data with conviction by knowing what to ignore, let us dig a little deeper:

Descriptive Data
Generally, demographic data—such as people’s income, age, number of children, housing size, dwelling type, occupation, etc.—fall under this category. For B-to-B applications, “Firmographic” data—such as number of employees, sales volume, year started, industry type, etc.—would be considered as descriptive data. It is about what the targets “look like” and, generally, they are frozen in the present time. Many prominent data compilers (or data brokers, as the U.S. government calls them) collect, compile and refine the data and make hundreds of variables available to users in various industry sectors. They also fill in the blanks using predictive modeling techniques. In other words, the compilers may not know the income range of every household, but using statistical techniques and other available data—such as age, home ownership, housing value, and many other variables—they provide their best estimates in case of missing values. People often have some allergic reaction to such data compilation practices siting privacy concerns, but these types of data are not about looking up one person at a time, but about analyzing and targeting groups (or segments) of individuals and households. In terms of predictive power, they are quite effective and results are very consistent. The best part is that most of the variables are available for every household in the country, whether they are actual or inferred.

Other types of descriptive data include geo-demographic data, and the Census Data by the U.S. Census Bureau falls under this category. These datasets are organized by geographic denominations such as Census Block Group, Census Tract, Country or ZIP Code Tabulation Area (ZCTA, much like postal ZIP codes, but not exactly the same). Although they are not available on an individual or a household level, the Census data are very useful in predictive modeling, as every target record can be enhanced with it, even when name and address are not available, and data themselves are very stable. The downside is that while the datasets are free through Census Bureau, the raw datasets contain more than 40,000 variables. Plus, due to the budget cut and changes in survey methods during the past decade, the sample size (yes, they sample) decreased significantly, rendering some variables useless at lower geographic denominations, such as Census Block Group. There are professional data companies that narrowed down the list of variables to manageable sizes (300 to 400 variables) and filled in the missing values. Because they are geo-level data, variables are in the forms of percentages, averages or median values of elements, such as gender, race, age, language, occupation, education level, real estate value, etc. (as in, percent male, percent Asian, percent white-collar professionals, average income, median school years, median rent, etc.).

There are many instances where marketers cannot pinpoint the identity of a person due to privacy issues or challenges in data collection, and the Census Data play a role of effective substitute for individual- or household-level demographic data. In predictive analytics, duller variables that are available nearly all the time are often more valuable than precise information with limited availability.

Transaction Data/Behavioral Data
While descriptive data are about what the targets look like, behavioral data are about what they actually did. Often, behavioral data are in forms of transactions. So many just call it transaction data. What marketers commonly refer to as RFM (Recency, Frequency and Monetary) data fall under this category. In terms of predicting power, they are truly at the top of the food chain. Yes, we can build models to guess who potential golfers are with demographic data, such as age, gender, income, occupation, housing value and other neighborhood-level information, but if you get to “know” that someone is a buyer of a box of golf balls every six weeks or so, why guess? Further, models built with transaction data can even predict the nature of future purchases, in terms of monetary value and frequency intervals. Unfortunately, many who have access to RFM data are using them only in rudimentary filtering, as in “select everyone who spends more than $200 in a gift category during the past 12 months,” or something like that. But we can do so much more with rich transaction data in every stage of the marketing life cycle for prospecting, cultivating, retaining and winning back.

Other types of behavioral data include non-transaction data, such as click data, page views, abandoned shopping baskets or movement data. This type of behavioral data is getting a lot of attention as it is truly “big.” The data have been out of reach for many decision-makers before the emergence of new technology to capture and store them. In terms of predictability, nevertheless, they are not as powerful as real transaction data. These non-transaction data may provide directional guidance, as they are what some data geeks call “a-camera-on-everyone’s-shoulder” type of data. But we all know that there is a clear dividing line between people’s intentions and their commitments. And it can be very costly to follow every breath you take, every move you make, and every step you take. Due to their distinct characteristics, transaction data and non-transaction data must be managed separately. And if used together in models, they should be clearly labeled, so the analysts will never treat them the same way by accident. You really don’t want to mix intentions and commitments.

The trouble with the behavioral data are, (1) they are difficult to compile and manage, (2) they get big; sometimes really big, (3) they are generally confined within divisions or companies, and (4) they are not easy to analyze. In fact, most of the examples that I used in this series are about the transaction data. Now, No. 3 here could be really troublesome, as it equates to availability (or lack thereof). Yes, you may know everything that happened with your customers, but do you know where else they are shopping? Fortunately, there are co-op companies that can answer that question, as they are compilers of transaction data across multiple merchants and sources. And combined data can be exponentially more powerful than data in silos. Now, because transaction data are not always available for every person in databases, analysts often combine behavioral data and descriptive data in their models. Transaction data usually become the dominant predictors in such cases, while descriptive data play the supporting roles filling in the gaps and smoothing out the predictive curves.

As I stated repeatedly, predictive analytics in marketing is all about finding out (1) whom to engage, and (2) if you decided to engage someone, what to offer to that person. Using carefully collected transaction data for most of their customers, there are supermarket chains that achieved 100 percent customization rates for their coupon books. That means no two coupon books are exactly the same, which is a quite impressive accomplishment. And that is all transaction data in action, and it is a great example of “Big Data” (or rather, “Smart Data”).

Attitudinal Data
In the past, attitudinal data came from surveys, primary researches and focus groups. Now, basically all social media channels function as gigantic focus groups. Through virtual places, such as Facebook, Twitter or other social media networks, people are freely volunteering what they think and feel about certain products and services, and many marketers are learning how to “listen” to them. Sentiment analysis falls under that category of analytics, and many automatically think of this type of analytics when they hear “Big Data.”

The trouble with social data is:

  1. We often do not know who’s behind the statements in question, and
  2. They are in silos, and it is not easy to combine such data with transaction or demographic data, due to lack of identity of their sources.

Yes, we can see that a certain political candidate is trending high after an impressive speech, but how would we connect that piece of information to whom will actually donate money for the candidate’s causes? If we can find out “where” the target is via an IP address and related ZIP codes, we may be able to connect the voter to geo-demographic data, such as the Census. But, generally, personally identifiable information (PII) is only accessible by the data compilers, if they even bothered to collect them.

Therefore, most such studies are on a macro level, citing trends and directions, and types of analysts in that field are quite different from the micro-level analysts who deal with behavioral data and descriptive data. Now, the former provide important insights regarding the “why” part of the equation, which is often the hardest thing to predict; while the latter provide answers to “who, what, where and when.” (“Who” is the easiest to answer, and “when” is the hardest.) That “why” part may dictate a product development part of the decision-making process at the conceptual stage (as in, “Why would customers care for a new type of dishwasher?”), while “who, what, where and when” are more about selling the developed products (as in “Let’s sell those dishwashers in the most effective ways.”). So, it can be argued that these different types of data call for different types of analytics for different cycles in the decision-making processes.

Obviously, there are more types of data out there. But for marketing applications dealing with humans, these three types of data complete the buyers’ portraits. Now, depending on what marketers are trying to do with the data, they can prioritize where to invest first and what to ignore (for now). If they are early in the marketing cycle trying to develop a new product for the future, they need to understand why people want something and behave in certain ways. If signing up as many new customers as possible is the immediate goal, finding out who and where the ideal prospects are becomes the most imminent task. If maximizing the customer value is the ongoing objective, then you’d better start analyzing transaction data more seriously. If preventing attrition is the goal, then you will have to line up the transaction data in time series format for further analysis.

The business goals must dictate the analytics, and the analytics call for specific types of data to meet the goals, and the supporting datasets should be in “analytics-ready” formats. Not the other way around, where businesses are dictated by the limitations of analytics, and analytics are hampered by inadequate data clutters. That type of business-oriented hierarchy should be the main theme of effective data management, and with clear goals and proper data strategy, you will know where to invest first and what data to ignore as a decision-maker, not necessarily as a mathematical analyst. And that is the first step toward making the Big Data smaller. Don’t be impressed by the size of the data, as they often blur the big picture and not all data are created equal.

McKinsey Thinks Bland, Generic Loyalty Programs Are Killing Business – And They May Be Right!

A recent Forbes article by McKinsey, “Making Loyalty Pay: Six Lessons From the Innovators,” showed loyalty program participation has steadily increased during the past five years (a 10 percent annual rate of growth), with the average household now having almost 25 memberships. For all of that growing popularity, there are huge questions for marketers: Are the programs contributing to increased sales? And what is the impact of loyalty programs on enterprise profitability?

A recent Forbes article by McKinsey, “Making Loyalty Pay: Six Lessons From the Innovators,” showed loyalty program participation has steadily increased during the past five years (a 10 percent annual rate of growth), with the average household now having almost 25 memberships. For all of that growing popularity, there are huge questions for marketers: Are the programs contributing to increased sales? And what is the impact of loyalty programs on enterprise profitability?

Overall, companies with loyalty programs have grown at about the same rate as companies without them; but there is variance in performance value among industries. These programs produce positive sales increases for hotels, for example, but negative sales impact on car rental, airlines and food retail. And, companies with higher loyalty program spend had lower margins than companies in the same sector which do not spend on high-visibility loyalty programs.

McKinsey has noted that, “Despite relative underperformance in terms of revenue growth and profitability, over the past five years, market capitalization for companies that greatly emphasize loyalty programs has outpaced that of companies that don’t.” This, as they see it, may be indicative of hope among companies with programs that long-term customer value can be generated.

Within the McKinsey report, several strategies are offered for helping businesses overcome the negatives often associated with loyalty programs. Key among these are:

  • Integrate Loyalty Into the Full Experience
    Companies can link the loyalty program into the overall purchase and use experience. An example cited in the article is Starbucks, which has created its program to reflect the uniqueness of its café experience. Loyalty is built into the program by integrating payments and mobile technology, which appeals to its target audience.
  • Use the Data
    This may be the most important opportunity represented by loyalty programs. Data collected from the programs can offer competitive opportunities. Tesco, the largest supermarket chain on the planet, has been doing loyalty program member number-crunching for years through DunnHumby. Similarly, Caesars Entertainment has rich databases on its high-rolling program members. One retailer has combined its loyalty program with a 5 percent point-of-sale discount, building volume from its highest-value customers. In another well-documented example, a retailer has used its loyalty program data to identify future mothers before other chains, thus targeting offers to capture both their regular spend and new category purchases as buying habits evolve.
  • Build Partnerships
    As stated on so many occasions, organizations that build trust generate stronger, more bonded, customer behavior. This applies to loyalty programs as well, where there is ample opportunity to build cross-promotion for customers with non-competing products and services. In the U.K., Sainsbury, the major supermarket competitor of Tesco, has partnered with Nectar, a major loyalty coalition. Nectar has more members than Tesco, and participants can collect rewards across a large number of non-competing retailers. Through partnership, Sainsbury’s offers customers a broader and deeper value proposition; and Nectar also generates data from coalition partners, which it uses to better target promotions to customers.
  • Solve Customer and Industry Pain Points
    Numerous customer behavior studies have shown that people will gravitate to, and pay more for, better service. A perfect example of this is Amazon Prime, where additional payment gets customers faster delivery and digital tracking. This is good for Amazon (estimates are that members spend more than four times more with Amazon than non-members), its customers, and its suppliers, who also get access to Prime customers and the positive rub-off of affiliating with a trusted brand.
  • Maximize Difference Between Perceived Value and Real Cost
    Often, program elements can represent high perceived value without adding much in the way of bottom-line cost to the sponsor. The example cited is Starwood Hotels and Resorts where, through its Starwood Preferred Guest (SPG) program, there is a focus on personal leisure travel rewards for high-spending frequent guests.
  • Allocate Loyalty Reinvestment to the Most Valuable Customers
    Many companies have only recently come to the realization that some customers are more valuable than others; and, to be successful, loyalty programs need to target the higher revenue customers. In 2010, Southwest Airlines revamped its loyalty program to make rewards more proportional to ticket price; and this has better targeted the most profitable customers, as well as enabled the airline to adopt a loyalty behavior metric that is closely tied to actual revenue generation.

Loyalty programs continue to grow, but they are also tending to become more closely integrated with brand-building and multichannel customer experience optimization. But, there is also lots of commoditization and passivity were these programs are concerned—sort of the “If You Build It, They Will Come” syndrome at work. And, of course, there’s a mini contra movement among some retail chains, where they have removed established loyalty programs—or never initiated them in the first place—in favor of everyday low prices and more efficient performance.

Too Big to Fail – But Not Too Big to Suck

On a recent “Real Time With Bill Maher” show, Maher responded to the announcement that Time Warner Cable would merge with Comcast Corp. in a $45 billion purchase. He noted that, combined, the two cable systems represent 19 of the 20 largest U.S. markets; and, apart from suppliers like Dish and DirecTV, they have no competitors in these metros. Further, Maher said, the two companies have the lowest customer satisfaction ratings of any cable system. So, as he asked his panelists, where is the value for customers in this merger if both companies are known to have questionable service performance?

On a recent “Real Time With Bill Maher” show, Maher responded to the announcement that Time Warner Cable would merge with Comcast Corp. in a $45 billion purchase. He noted that, combined, the two cable systems represent 19 of the 20 largest U.S. markets; and, apart from suppliers like Dish and DirecTV, they have no competitors in these metros. Further, Maher said, the two companies have the lowest customer satisfaction ratings of any cable system. So, as he asked his panelists, where is the value for customers in this merger if both companies are known to have questionable service performance?

The Federal Communications Commission (FCC) will, of course, have a great deal to say about whether this merger goes through or not. During the past couple of decades, we’ve seen a steady decline in the number of cable companies, from 53 at one point to only six now. Addressing some of the early negative reaction to its planned purchase of TWC-which would increase Comcast’s cable base to 30 million subscribers from the 22 million it currently has (a bit less than 30 percent of the overall market)-Comcast has already stated that it will make some concessions to have the merger approved. But, that said, according to company executives, the proposed cost savings and efficiencies that will “ultimately benefit customers” are not likely to either reduce monthly subscription prices or even cause them to rise less rapidly.

Comcast executives have stated that the value to consumers will come via “quality of service, by quality of offerings and by technological innovations.” David Cohen, their Executive VP, said: “Putting these two companies together will not deprive a single customer in America of a choice he or she will have today.” (Opens as a PDF) He also said, “I don’t believe there’s any way to argue that consumers are going to be hurt from a price perspective as a result of this transaction.” But, that said, he also admitted, “Frankly, most of the factors that go into customer bills are beyond our control.” Not very encouraging.

As anyone remotely familiar with Comcast’s history will understand, this is not the first time the company has navigated the river of communications company consolidation: 1995, Scripps, 800,000 subscribers, 1998, Jones Intercable, 1.1 million subscribers; 2000, Lenfest Communications, 1.3 million subscribers.

In 2002, Comcast completed acquisition of AT&T Broadband, in a deal worth $72 billion. This increased the company’s base to its current level of 22 million subscribers, and gave it major presence in markets like Atlanta, Boston, Chicago, Dallas-Ft. Worth, Denver, Detroit, Miami, Philadelphia and San Francisco-Oakland. In a statement issued by Comcast at the time the purchase was announced, again there was a claim that the merger with AT&T would benefit all stakeholders: “Combining Comcast with AT&T Broadband is a once in a lifetime opportunity that creates immediate value and positions the company for additional growth in the future. Shareholders, employees, and customers alike are poised to reap considerable benefits from this remarkable union.”

There have been technological advances, additional content, and enhanced service, during the ensuing 13 years. But “immediate value” and “considerable benefits”? Having been professionally involved with customer research conducted at the time of this merger, there was genuine question regarding the value perceived by the newly acquired AT&T customers. In a study among customers who discontinued with Comcast post-merger, and also among customers who had been Comcast customers or AT&T customers prior to the merger, poor picture quality (remember, these were the days well before HD), service disruption and high/continually rising prices were the key reasons given for defection to a competitor.

Conversely, when asked to rate their current suppliers on both key attribute importance (a surrogate measure of performance expectation) and performance itself, the highest priorities were all service-related:

  • Reliability of cable service
  • Availability of customer service when needed
  • Speed of service problem resolution
  • Responsiveness of customer service staff

On all principal service attributes except “speed of service problem resolution,” the new supplier was given higher ratings than either Comcast or AT&T. And there were major gaps in all of the above areas. Overall, close to 90 percent of these defected customers said they would be highly likely to continue the relationship with their new supplier. When correlation analysis was performed, pricing and service performance were the key driving factors. In addition, even if Comcast were now able to offer services that overcame their reasons for defection, very few (only about 10 percent) said they would be willing to become Comcast customers again.

Finally, we’ve often focused on unexpressed and unresolved complaints as leading barometers, or indicators, of possible defection. Few of the customers interviewed indicated problems with their current suppliers; however, as in other studies, problem and complaint issues were frequently surfaced for both Comcast and AT&T.

It should be noted that having lost a significant number of customers to Verizon’s FiOS, Comcast has a winback program under way, leveraging quotes from subscribers who have returned to the Xfinity fold. In the usual Macy’s/Gimbel’s customer acquisition and capture theater of war, this marks a marketing change for Comcast. As often observed (and even covered in an entire book, with my co-author, consultant Jill Griffin), winback marketing strategies are rather rarely applied, but can be very successful.

One of the key consumer concerns, especially as it may impact monthly bills, is the cost and control of content. For example, Netflix has agreed to pay Comcast for an exclusive direct connection into its network. As one media analyst noted, “The largest cable company in the nation, on the verge of improving its power to influence broadband policy, is nurturing a class system by capitalizing on its reach as a consumer Internet service provider (ISP).” This could, John C. Abell further stated, be a “game-changer.” Media management and control such as this has echoes of Big Brother for customers, and it is all the more reason Comcast should be paying greater attention to the evolving needs, as well as the squeeze on wallets, of its customers.

Perhaps the principal lesson here, assuming that the FCC allows this merger to proceed and ultimately consummate, will be for Comcast to be proactive in building relationships and service delivery. There’s very little that will increase consumer trust more than “walking the talk,” delivering against the claims of what benefits customers will stand to receive. Conversely, there’s little that will undermine trust and loyalty faster, and more thoroughly, than underdelivery on promises.

What Customer-Centric, Customer-Obsessed Companies Must Do

In building relationships with and value for customers, my longtime observation is most organizations tend to progress through several stages of performance: customer awareness, customer sensitivity, customer focus and customer obsession. Here is the “executive summary” version of some conditions of each stage.

In building relationships with and value for customers, my longtime observation is most organizations tend to progress through several stages of performance: customer awareness, customer sensitivity, customer focus and customer obsession.

Here is the “executive summary” version of some conditions of each stage.

Customer Awareness
Customers are known, but in the aggregate. The organization believes it can select its customers and understand their needs. Measurement of performance is rudimentary, if it exists at all; and customer data are siloed. There’s a traditional, hierarchical, top-down management model, with “chimneyed” or “smokestack” communication (goes up or down, but not horizontal) with little evidence of teaming.

Customer Sensitivity
Customers are known, but still mostly in the aggregate. Customer service is somewhat more evident (though still viewed as a cost center), with a focus on complaint and problem resolution (but not proactive complaint generation; internal groups tend to point fingers and blame each other for negative customer issues). Measurement is mostly around customer attitudes and functional transactions, i.e. satisfaction, with little awareness of emotional relationship drivers. The organization has a principally traditional, hierarchical, top-down management model, with “chimneyed” or “smokestack” communication (goes up or down, but not horizontal), with some evidence of teaming (mostly in areas of complaint resolution).

Customer Focus
Customers are both known and valued, down to the individual level, and they are recognized as having different needs, both functional and emotional. The customer life cycle is front-and-center; and performance measurement is much more about emotion and value drivers than satisfaction. Service and value provision is regarded as an enterprise priority; and customer stabilization and recovery are goals when problems or complaints arise. Communication and collaboration with customers, between employees, and between employees and customers is featured. Management model and style is considerably more horizontal, with greater emphasis on teaming to improve customer value processes.

It’s notable that, at this more evolved and advanced stage of enterprise customer-centricity, complaints are thought of more in terms of a life cycle component, and recovery is more of a strategy than a resolution.

Customer Obsession
Throughout the organization, customer needs and expectations—especially those that are emotional—are well understood, and response is appropriate (and often proactive).

Everyone is involved in providing value to customers—from C-suite to front-line—and everyone understands his/her role. Customer behavior is recognized as essential to enterprise success, and optimal relationships are sought.

Performance measurement is focused, and shared, on what most monetizes customer behavior (loyalty, emotion and communication metrics—such as brand-bonding and advocacy—replace satisfaction and recommendation).

Customer service (along with pipelines and processes) is an enterprise priority, and seen as a vital, and profitable, element of value delivery.

The management model is far more horizontal, replacing traditional hierarchy; and there is an emphasis on teaming and inclusion of customers to create or enhance value.

Companies that are customer-obsessed, and what makes them both unique and successful, have been extensively profiled by consultants and the business press. Often, they go so far as to create emotionally driven, engaged and even branded experiences for their customers, strategically differentiating them from their peers.

In addition, these companies focus on the complete customer life cycle, and much more on retention, loyalty and risk mitigation (and even winback) than acquisition. Support experiences are strategic, nimble and seamless, and often omnichannel. Multiple sources of data are used to develop insights. Recognizing the information needs of their customers, they invest in altruistic content creation (over advertising); and they communicate proactively and in as personalized a manner as possible

Customer obsession, what I refer to as “inside-out” customer-centricity, has been a frequent subject of my blogs and articles: One of Albert Einstein’s iconic quotes reflects the complete dedication of resources and values needed for an organization to optimize its relationships with customers: “Only one who devotes himself to a cause with his whole strength and soul can be a true master.” Mastery requires, as well, a storehouse of experience coming from experimentation; so, just like in the pole vault and high jump, we can expect that the customer-centricity bar will continue to be raised.

Building Customer-Centric, Trust-Based Relationships

More than a buzzword, “being human,” especially in brand-building and leveraging customer relationships, has become a buzz-phrase or buzz-concept. But, there is little that is new or trailblazing in this idea. To understand customers, the enterprise needs to think in human, emotional terms. To make the brand or company more attractive, and have more impact on customer decision-making, there must be an emphasis on creating more perceived value and more personalization. Much of this is, culturally, operationally, and from a communications perspective, what we have been describing as “inside-out advocacy” for years.

More than a buzzword, “being human,” especially in brand-building and leveraging customer relationships, has become a buzz-phrase or buzz-concept. But, there is little that is new or trailblazing in this idea. To understand customers, the enterprise needs to think in human, emotional terms. To make the brand or company more attractive, and have more impact on customer decision-making, there must be an emphasis on creating more perceived value and more personalization. Much of this is, culturally, operationally, and from a communications perspective, what we have been describing as “inside-out advocacy” for years.

Most brands and corporations get by on transactional approaches to customer relationships. These might include customer service speed, occasional price promotions, merchandising gimmicks, new product offerings, and the like. In most instances, the customers see no brand “personality” or brand-to-brand differentiation, and their experience of the brand is one-dimensional, easily capable of replacement. Moreover, the customer has no personal investment in choosing, and staying with, one brand or supplier over another.

A key opportunity for companies to become stronger and more viable to customers is creation of branded experiences. Beyond simply selling a product or service, these “experiential brands” connect with their customers. They understand that delivering on the tangible and functional elements of value are just tablestakes, and that connecting and having an emotionally based relationship with customers is the key to leveraging loyalty and advocacy behavior.

These companies are also invariably quite disciplined. Every aspect of a company’s offering—customer service, advertising, packaging, billing, products, etc.—are all thought out for consistency. They market, and create experiences, within the branded vision. IKEA might get away with selling super-expensive furniture, but it doesn’t. Starbucks might make more money selling Pepsi, but it doesn’t. Every function that delivers experience is “closed-loop,” carefully maintaining balance between customer expectations and what is actually executed.

In his 2010 book, “Marketing 3.0: From Products to Customers to the Human Spirit,” noted marketing scholar Philip Kotler recognized that the new model for organizations was to treat customers not as mere consumers, but as the complex, multi-dimensional human beings they are. Customers, in turn, have been choosing companies and products that satisfy deeper needs for participation, creativity, community and idealism.

This sea change is why, according to Kotler, the future of marketing lies in creating products, services and company cultures that inspire, include and reflect the values of target customers. It also meant that every transaction and touchpoint interaction, and the long-term relationship, needed to carry the organization’s unique stamp, a reflection of the perceived value represented to the customer.

Kotler picked up a theme that was articulated in the 2007 book, “Firms of Endearment.” Authors Jagdish N. Sheth, Rajendra S. Sisodia and David B. Wolfe called such organizations “humanistic” companies, i.e. those which seek to maximize their value to each group of stakeholders, not just to shareholders. As they state, right up front (Chapter 1, Page 4):

“What we call a humanistic company is run in such a way that its stakeholders—customers, employees, suppliers, business partners, society, and many investors—develop an emotional connection with it, an affectionate regard not unlike the way many people feel about their favorite sports teams. Humanistic companies—or firms of endearment (FoEs)—seek to maximize their value to society as a whole, not just to their shareholders. They are the ultimate value creators: They create emotional value, experiential value, social value, and, of course, financial value. People who interact with such companies feel safe, secure, and pleased in their dealings. They enjoy working with or for the company, buying from it, investing in it, and having it as a neighbor.”

For these authors, a truly great company is one that makes the world a better place because it exists. It’s as simple as that. In the book, they have identified about 30 companies, from multiple industries, that met their criteria. They included CarMax, BMW, Costco, Harley-Davidson, IKEA, JetBlue, Johnson & Johnson, New Balance, Patagonia, Timberland, Trader Joe’s, UPS, Wegmans and Southwest Airlines. Had the book been written a bit later, it’s likely that Zappos would have made their list, as well.

The authors compared financial performance of their selections with the 11 public companies identified by Jim Collins in “Good to Great” as superior in terms of investor return over an extended period of time. Here’s what they learned:

  • Over a 10-year horizon, their selected companies outperformed the “Good to Greatcompanies by 1,028 percent to 331 percent (a 3.1 to 1 ratio)
  • Over five years, their selected companies outperformed the “Good to Great companies by 128 percent to 77 percent (a 1.7 to 1 ratio)

Just on the basis of comparison to the Standard & Poor’s 500 index, the public companies singled out by “Firms of Endearment” returned 1,026 percent for investors during the 10 years ending June 30, 2006, compared to 122 percent for the S&P 500—more than an 8 to 1 ratio. Over 5 years, it was even higher—128 percent compared to 13 percent, about a 10 to 1 ratio. Bottom line: Being human is good for the balance sheet, as well as the stakeholders.

Exemplars of branded customer experience also understand that there is a “journey” for customers in relationships with preferred companies. It begins with awareness, how the brand is introduced, i.e. the promise. Then, promise and created expectations must at least equal—and, ideally, exceed—real-world touchpoint results (such as through service), initially and sustained over time, with a minimum of disappointment.

As noted, there is a strong recognition that customer service is especially important in the branded experience. Service is one of the few times that companies will directly interact with their customers. This interaction helps the company understand customers’ needs while, at the same time, shaping customers’ overall perception of the company and influencing both downstream communication and future purchase.

And, branding the customer experience requires that the brand’s image, its personality if you will, is sustained and reinforced in communications and in every point of contact. Advanced companies map and plan this out, recognizing that experiences are actually a form of branding architecture, brought to life through excellent engineering. Companies need to focus on the touchpoints which are most influential.

Also, how much influence do your employees have on customer value perceptions and loyalty behavior through their day-to-day interactions? All employees, whether they are customer-facing or not, are the key common denominator in delivering optimized branded customer experiences. Making the experience for customers positive and attractive at each point where the company interacts with them requires an in-depth understanding of both customer needs and what the company currently does to achieve that goal, particularly through the employees. That means companies must fully comprehend, and leverage, the impact employees have on customer behavior.

So, is your company “human”? Does it understand customers and their individual journeys? Are customer experiences “human” and branded? Is communication, and are marketing efforts, micro-segmented and even personalized? Does the company create emotional, trust-based connections and relationships with customers? If the answer to these questions is “YES,” then “being human” becomes a reality, the value of which has been recognized for some time, and not merely as a buzz-concept.

Marketing Data: Do I Own My Own Name?

I’ve always been uncomfortable with the position taken by some privacy advocates that each of us owns our own information—and thus has some form of property rights derived from this information—and that marketers shouldn’t have use of that information without first having permission and providing compensation

I’ve always been uncomfortable with the position taken by some privacy advocates that each of us owns our own information—and thus has some form of property rights derived from this information—and that marketers shouldn’t have use of that information without first having permission and providing compensation. To this, I say—hey OK, but let’s be pragmatic.

Certainly, if I’m a celebrity, where my name and likeness has commercial value, perhaps as an endorsement, such an “ownership” rationale is a valid one.

But in the exchange of customer data for marketing purposes, this argument lacks merit, in my opinion. The value of my name on a mailing list, for example—mail, email, telephone, otherwise—has nothing to do with “my” name being on the list or, for that matter, “your” name being on that same list. (Even when we are both see ourselves as celebrities.)

Rather, the value of both our names being on the same list is by knowing the shared attribute that placed us both there—alongside the thousands of others on that list. In the world of response lists, it’s the sweat equity of the business where you and I both chose to become a customer that deserves the compensation in any data transaction, as it alone built the list by building a business where you and I both chose to become customers.

Yes, that marketer must provide notice, choice, security, sensitivity, marketing data for marketing use only, and perform other ethical obligations that are part of the self-regulatory process that have governed this business for nearly 50 years—recognizing that customer data is our most important asset, and that consumer trust and acceptance serves as the foundation of the data-driven marketing field. Privacy policies, preference centers, in-house suppressions and DMAchoice collectively serve the consumer empowerment process by enabling transparency and control in this data exchange.

In the world of compiled lists, where third parties assemble observed data for marketing purposes, again there is the sweat equity of the entities assembling and analyzing that data to “create” or “discover” the shared attributes of that data. Knowing these attributes is where the combined data derive their value. Marketers deploy activity based on these attributes to generate commerce. While the relationship between individuals and these third parties may be indirect, we still have the same ethical codes and opt-out tools governing the process. Recently, in the case of Acxiom, we’ve seen such a data compiler working to establish a direct relationship with consumers, providing individuals with the ability to inspect the data the company holds and to suggest corrections—as if the firm were a (highly regulated) credit bureau. (It is not.)

The fact that my name—Chet Dalzell—is on both response and compiled lists, to me, doesn’t entitle me to anything except to expect and demand that these movers of data act as responsible stewards of this information using well established ethics and self-regulatory methods. (Granted, in the US, there are legal requirements that must be met in such sensitive areas as credit, personal finance, health and children’s data.)

This flow of data, as the Direct Marketing Association most recently reaffirmed, generates huge social and economic value—and, in my view, my own participation as a customer in the marketplace is my agreement to allow such data exchange to happen. In fact, were it not for such flows, I might never have been provided an opportunity to become a customer in the first place. Benefits to consumers accumulate, while harm is nowhere part of the marketing ecosystem—other than to protect from identity theft and fraud. I find it fascinating some would-be regulators fail to grasp this truth.

That’s why inflexible government regulations—and opt-in-only regimes—and technology strictures that interfere with my interaction with brands are so troublesome. Such restrictions may claim to be about privacy; more often than not, they’re really motivated by political grand-standing, anti-competitive business models, and the forced building of new data siloes that do nothing to advance consumer protection—and potentially ruin data-driven marketing.

Yes, I own my name—and by choosing to be a customer of your brand, so do you own your customer list. Of course, I am the ultimate regulator in this process. For whim or reason, I can choose to take my business elsewhere.

Now, what about my Twitter, Facebook, Google and Yahoo! profiles?

When Companies Lose Customers …

United Parcel Service suffered staggering customer defection as a consequence of its 15-day Teamsters work stoppage in 1997. The result was that, even after their 80,000 drivers were back behind the wheels of their delivery trucks or tractor-trailers, many thousands of UPS workers were laid off. A UPS manager in Arkansas was quoted as saying: “To the degree that our customers come back will dictate whether those jobs come back.”

United Parcel Service suffered staggering customer defection as a consequence of its 15-day Teamsters work stoppage in 1997. The result was that, even after their 80,000 drivers were back behind the wheels of their delivery trucks or tractor-trailers, many thousands of UPS workers were laid off. A UPS manager in Arkansas was quoted as saying: “To the degree that our customers come back will dictate whether those jobs come back.”

The UPS loss was a gain for Federal Express, Airborne, RPS and even the United States Postal Service. They provided services during the strike that made UPS’ customers see the dangers of using a single delivery company to handle their packages and parcels. FedEx, for example, reported expecting to keep as much as 25 percent of the 850,000 additional packages it delivered each day of the strike.

UPS’ customer loss woes and the impact on its employees was a very public display of the consequences of customer turnover. Most customer loss is relatively unseen, but it has been determined that many companies lose between 10 percent and 40 percent of their customers each year. Still more customers fall into a level of dormancy, or reduced “share of customer” with their current supplier, moving their business to other companies, thus decreasing the amount they spend with the original supplier. The economic impact on companies, not to mention the crushing moral effect on employees—downsizing, rightsizing, plant closings, layoffs, etc.—are the real effects of customer loss.

Lost jobs and lost profits propelled UPS into an aggressive win-back mode as soon as the strike was settled. Customers began receiving phone calls from UPS officials assuring them that UPS was back in business, apologizing for the inconvenience and pledging that their former reliability had been restored. Drivers dropping by for pick-ups were cheerful and confident, and they reinforced that things were back to normal. UPS issued letters of apology and discount certificates to customers to further help heal the wounds and rebuild trust. And face-to-face meetings with customers large and small were initiated by UPS—all with the goal of getting the business back.

These win-back initiatives formed an important bridge of recovery back to the customer. And it worked. The actions, coupled with the company’s cost-effective services, continuing advances in shipping technology, and the dramatic growth of online shopping, enabled UPS to reinstate many laid off workers while increasing its profits a remarkable 87 percent in the year following the devastating strike.

UPS is hardly an isolated case. Protecting customer relationships in these uncertain times is a fact of life for every business. We’ve entered a new era of customer defection, where customer churn is reaching epidemic proportions and is wrecking businesses and lives along the way. It’s time to truly understand the consequences of customer loss and, in turn, apply proven win-back strategies to regain these valuable customers.

Nowhere are the effects of customer defection more visible than in the world of Internet and mobile commerce, where the opportunities for customer loss occur at warp speed. E-tailers and Web service companies are spending incredible sums of money to draw customers to their sites, and to modify their messages and images so that they are compatible and user-friendly on all devices. Because of this, relatively few of these companies, including many well-established sites, have turned a profit. Customer loss (and lack of recovery) is a key contributor. E-customers have proven to be a high-maintenance lot. They want value, and they want it fast. These customers show little tolerance for poor Web architecture and navigation, difficult to read pages, and outdated information or insufficient customer service. Expectations for user experience are very high, and rising rapidly.

Internet and mobile customers, to be sure, have some of the same value delivery needs as brick-and-mortar customers; but, they are also different from brick-and-mortar customers in many important and loyalty-leveraging respects. They are more demanding and require much more contact. They require multi-layer benefits, in the form of personalization, choice, customized experience, privacy, current information, competitive pricing and feedback. They want partnering and networking opportunities. When site download times are too long, order placement mechanisms too cumbersome, order acknowledgment too slow, or customer service too overwhelmed to respond in a timely fashion, online shoppers will quickly abandon their purchase transactions or not repeat them. Further, they are highly unlikely to return to a site which has caused negative experiences.

What’s more, the new communication channels also serve as a high-speed information pathway for negative customer opinion. If unhappy customers in the brick-and-mortar world usually express their displeasure to between two and 20 people, on the Internet, angry former customers have the opportunity to impact thousands more. There are now scores of sites offering similar negative messages about companies in many industries, and giving customers, and even former employees, a place to express grievances. It’s a new form of angry former customer sabotage, which adds to the economic and cultural effect of customer turnover.

For many of these sites, part of their charter is to help consumers find value; and, like us, they understand that customers will provide loyalty in exchange for value. They also recognize that the absence of value drives customer loss, and that insufficient or ineffective feedback handling processes can create high turnover. As one states: “The Internet is the most consumer-centric medium in history—and we will help consumers use it to their greatest personal advantage. We will increase the influence of individuals through networks of millions. We will raise the stakes for companies to respond. We will require companies to respect consumers’ choice, privacy and time, and will expose those that do not.” This may sound a bit like Orwell’s “Animal Farm,” but it does acknowledge the power of negative, as well as positive, customer feedback.

Some businesses seem minimally concerned about losing a customer; but the only thing worse than the loss of high value customers is neglecting the opportunity to win them back. When customer lifetime value is interrupted, it often makes both economic and cultural sense for the company to make an active, serious effort to recover them. This is true for both business-to-business and consumer products or services.

So how does a company defend itself against the perils of customer loss? The best plan, of course, is a proactive one that anticipates customer defection and works hard to lessen the risk. Companies need defection-proofing strategies, including intelligent gathering and application of customer data, the use of customer teams, creating employee loyalty, engagement and ambassadorship, and the basic strategy of targeting the right kind of customers in the first place. But in today’s hyper-competitive marketplace, no retention or relationship program is complete without a save and win-back component. There is mounting evidence that the probability of win-back success and the benefits surrounding it far outweigh the investment costs. Yet, most companies are largely unprepared to address this opportunity. It’s costing them dearly, and even driving them out of business.

Building and sustaining customer loyalty behavior is harder than ever before. Now is the time to put in place specific strategies and tools for winning back lost customers, saving customers on the brink of defection and making your company defection-proof.

Wearable Mobile Devices Are the New Black

This year’s hot trend in fashion is computers. Whether at SXSW or in the tech and media hubs on the coasts, people are excited about the watches, wristbands and “eyeframes” that double as computers. Not all of these gadgets will succeed and those that do probably will evolve rapidly from today’s versions. But the trend is real—and marketers need to take note. They can expect consumers open to new forms of discovery and deeper relationships with brands, but also who have less tolerance for advertising that’s irrelevant, disruptive or disrespectful of privacy.

This year’s hot trend in fashion is computers. Whether at SXSW or in the tech and media hubs on the coasts, people are excited about the watches, wristbands and “eyeframes” that double as computers. Not all of these gadgets will succeed and those that do probably will evolve rapidly from today’s versions. But the trend is real—and marketers need to take note. They can expect consumers open to new forms of discovery and deeper relationships with brands, but also who have less tolerance for advertising that’s irrelevant, disruptive or disrespectful of privacy.

Nothing exemplifies the widespread interest in wearable computers better than Pebble, a watch that has its own Internet interface, apps and waiting list of fans eager to buy it. Last year, the founders of Pebble went to the crowdsourcing site Kickstarter with just a vague business plan and raised $10 million from thousands of investors. In less than a year, Pebble started to ship product and, in the past month, has released programming guidelines for outside developers. Not to be outdone by a start-up, Apple, Google, Samsung and LG are all rumored to be working on smartwatches, and Nike has made a big splash with its own wristband that tracks calories burned—the Fuel Band. Probably the most ambitious of all is Google Glass, the smartphone/eyeglass hybrid that projects information directly onto the lens of the wearer. Initial versions for developers have begun to ship already.

All of these devices will take the mobile revolution to a new level. The original iPhone ushered in an era when consumers expect to receive relevant answers any time, anywhere, to any question—even if they haven’t asked it yet. Still, wearable computing adds another layer of complexity. With screens that are always on and always feeding information, there’s even less of a margin for error with irrelevant advertising, and more opportunity for location-specific discovery. There will be new types of data—e.g., biometrics, location, eye movements—that could be incredibly relevant to marketers, but also frightening for consumers already worried about personal privacy. As a result, most marketing opportunities will have to be truly opt-in and transparent in how data will be used—and how that use is actually a service.

Take Google Now, a service that lets users receive pertinent time-sensitive or location-sensitive information without asking for it. It’s currently on phones, but it’s ideally suited for Google Glass. Although Now has high use-value, there’s also a high potential for creepiness, something Baris Guletkin, co-creator of Now, understands: “We take privacy very seriously, and make it very clear what the user will get, and what kind of data we’ll be using, and lots of controls so they can turn things off that they don’t like.” Google is banking on the fact that a lot of people will make that tradeoff in order to get useful information on-the-go. If I’ve just landed in Paris on an overnight flight and I am walking to a meeting, I’m OK with Google knowing what type of food I like if that information is used to suggest boulangeries along my route with highly rated croissants. But not everyone will feel that way.

Current discovery engines, such as Yelp and Foursquare, could probably also make a relatively easy transition to something like Google Glass or evolved versions of a smartwatch. Other marketers, however, will have to create new ways to use personal data and tags within physical objects to provide information that’s pertinent and enhances a real-world experience, not interrupts it. Peter Dahlstrom and David Edelman of McKinsey have written a great article about “on-demand marketing,” They describe a scenario where a headset has an NFC chip that communicates with a smartphone and opens an app that shows the headset in different colors and has related offers. Combined with augmented reality on Google Glass, the possibilities for this type of technology are pretty exciting. Even if Glass doesn’t catch on with the mainstream population, it will likely spur innovation that will trickle down to smartphones.

In addition to discovery, a second transformative role for wearable computers may be in how they turn solitary offline activities into daily social activities, creating a durable bond with the brand.

Nike’s Fuel Band is a great example. Nike has taken the daily workout and turned into a shared activity. The wristband uses a motion detector to calculate the amount of calories a person is burning during the day and tracks it against personal goals. It also connects to an app that shares this information with friends, creating value by turning the fuel points into shared successes and, for some, a competition. Because it’s always on, it creates dozens, even hundreds, of daily touchpoints with the brand.

Fuel fully aligns the brand with staying in shape, a high value for many people, and the core need that its other products satisfy. Eventually, Nike could connect Fuel points to support public causes, which would align the brand with the core values of the “new consumer,” described by sustainable branding agency, BBMG,

“Thirty percent of the U.S. adult population—some 70 million consumers—New Consumers—are values-aspirational, practical purchasers who are constantly looking to align their actions with their ideals; yet tight budgets and time constraints require them to make practical trade-offs every day … To deliver on total value, it’s no longer about pushing products, it’s about creating platforms for ideas and experiences that help people live healthier, greener and better.”

The Fuel Band and competitors like Jawbone are such platforms. They don’t just turn offline activities into online, social ones, they also link the brand to the values of the customer.

The Fuel Band right now is one of the first wearable computers that has been a commercial success, because it enhances existing activities in innovative ways. We’ll soon see whether Glass, Pebble and others have similar levels of success. Regardless, we’ll continue to see new wearable computers down the line, and they will undoubtedly lead to new opportunities for marketers that are impossible to see today.