Know What to Automate With Machine Learning

There are many posers in the data and analytics industry. Unfortunately, some of them occupy managerial positions, making critical decisions based on superficial knowledge and limited experiences. I’ve seen companies wasting loads of money and resources on projects with no substantial value — all because posers in high places bought into buzzwords or false promises. As if buzzwords have some magical power to get things done “auto-magically.”

I’ve written articles about how to identify posers and why buzzwords suck. But allow me to add a few more thoughts, as the phrase “Machine Learning” is rapidly gaining that magical power in many circles. You’d think that machines could read our minds and deliver results on their own. Sorry to break it to you, but even in the world of Star Trek, computers still wouldn’t understand illogical requests.

Beware of people who try to employ machine learning and no other technique. Generally, such people don’t even understand what they are trying to automate, only caring about the cost reduction part. But the price that others end up paying for such a bad decision could be far greater than any savings. The worst-case scenario is automating inadequate practices, which leads to wrong places really fast. How can anyone create a shortcut if he doesn’t know how to get to the destination in the first place, or worse, where the destination is supposed to be?

The goal of any data project should never be employing machine learning for the sake of it. After all, you wouldn’t respect a guitarist who can’t play a simple lick, just because he has a $5,000 custom guitar on his shoulder.

Then, what is the right way to approach this machine learning hype? First, you must recognize that there are multiple steps in predictive modeling. Allow me to illustrate some major steps and questions to ask:

  1. Planning: This critical step is often the most difficult one. What are you trying to achieve through data and analytics? Building the most eloquent model can’t be the sole purpose outside academia. Converting business goals into tangible solution sets is a project in itself. What kind of analytics should be employed? What would be the outcome? How will those model scores be applied to actual marketing campaigns? How will the results would be measured? Prescribing proper solutions to business challenges within the limitation of systems, toolsets, and the budget is one of the most coveted skill sets. And it has nothing to do with tools like machine learning, yet.
  2. Data Audit: Before we chart a long analytics journey, let’s put a horse before the cart, as data is the fuel for an engine called machine learning. I’ve seen too many cases where the cart is firmly mounted before the horse. What data are we going to use? From what sources? Do we have enough data to perform the task? How far in time do the datasets go back? Are they merged in one place? Are they in usable forms? Too many datasets are disconnected, unstructured, uncategorized, and unclean. Even for the machines.
  3. Data Transformation: Preparing available data for advanced analytics is also a project in itself. Be mindful that you don’t have to clean everything; just deal with the elements that are essential for required analytics to meet pre-determined business goals. At this stage, you may employ machine learning to categorize, group, or reformat data variables. But note that such modules are quite different from the ones for predictions.
  4. Target Definition: Setting up proper model targets is half-art/half-science. If the target is hung on a wrong spot, the resultant model will never render any value. For instance, if you are targeting so-called “High Value” customers, how would you express it in mathematical terms? It could be defined by any combinations of value, frequency, recency, and product categories. The targets are to be set after a long series of assumptions, profiling, and testing. No matter what modeling methodology eventually gets employed, you do NOT want targets to be unilaterally determined by a machine. Even with a simple navigator, which provides driving directions through machine-based algorithms, the user must provide the destination first. A machine cannot determine where you need to go (at least not yet).
  5. Universe Definition: In what universe will the resultant model be applied and used? Model comparison universe is as important as the target itself, as a model score is a mathematical expression of differences between two dichotomous universes (e.g., buyers vs. non-buyers). Even with the same target, switching the comparison universe would render completely different algorithms. On top of that, you may want to put extra filters by region, gender, customer type, user segment, etc. A machine may determine distinct sets of universes that require separate models, but don’t relinquish all controls to machines, either. Machine may not aware of where you would apply the model.
  6. Modeling: This statistical work is comprised of sub-steps such as variable selection, variable transformation, binning, outlier exclusion, algorithm creation, and validation, all in multiple iterations. It is indeed laborious work, and “some” parts may be done by the machines to save time. You may have heard of terms such as Deep Learning, Neural Net, logistic regression, stepwise regression, Random Forest, CHAID analysis, tree analysis, etc. Some are to be done by machines, and some by human analysts. All those techniques are basically to create algorithms. In any case, some human touch is inevitable regardless of employed methodology, as nothing should be released without continuous testing, validation, and tweaking. Don’t blindly subscribe to terms like “unsupervised learning.”
  7. Application: An algorithm may have been created in a test environment, but to be useful, the model score must be applied to the entire universe. Some toolsets provide “in-database-scoring”, which is great for automation. Let me remind you that most errors happen before or after the modeling step. Again, humans should not be out of the loop until everything becomes a routine, all the way to campaign execution and attribution.
  8. Maintenance: Models deteriorate and require scheduled reviews. Even self-perpetuating algorithms should be examined periodically, as business environments, data quality, and assumptions may take drastic turns. The auto-pilot switch shouldn’t stay on forever.

So, out of this outline for a simple target modeling (for 1:1 marketing applications), which parts do you think can fully be automated without any human intervention? I’d say some parts of data transformation, maybe all of modeling, and some application steps could go on the hands-free route.

The most critical step of all, of course, is the planning and goal-setting part. Humans must breathe their intention into any project. Once things are running smoothly, then sure, we can carve out the parts that can be automated in a step-wise fashion (i.e., never in one shot).

Now, would you still believe sales pitches that claim all your marketing dreams will come true if you just purchase some commercial machine-learning modules? Even if decent toolsets are tuned up properly, don’t forget that you are supposed to be the one who puts them in motion, just like self-driving cars.

How to Integrate AI Tech Into Each Step of the Customer Journey

The Customer Lifecycle. The Sales Funnel. The Buyer’s Journey. All of these phrases are similar expressions of the same thing. They’re used to describe the process that it takes for a visitor to become a customer.

The Customer Lifecycle. The Sales Funnel. The Buyer’s Journey. All of these phrases are similar expressions of the same thing. They’re used to describe the process that it takes for a visitor to become a customer.

While the models and names of stages may have changed through the years, many agree that it can be boiled down to four simple components:

Awareness > Consideration > Decision > Loyalty

The No. 1 goal for most businesses is to generate more conversions (which primarily consists of sales). This can be through their marketing efforts, sales tactics, brand communication, conversion rate optimization, and other methods. Of late, many companies have developed critical competencies in using AI to nudge customers towards sales, and have improved their numbers drastically as a result.

AI, machine learning, and big data technology can all work hand-in-hand to improve the customer experience and support an optimized customer journey, which leads to more conversions in several key ways.

Let’s talk about how you can start using AI tech in each stage of the funnel.

Awareness

Marketing strategies these days are often heavily focused on the top of the funnel to build brand awareness and attract new customers. For many businesses, recognition is nearly equivalent to the value of their brand. Elena Veselinova and Marija Gogova Samonikov explain in their book Building Brand Equity and Consumer Trust Through Radical Transparency Practices that brand impact is a continuous process that insures purchases, cash flow, revenue and share value. Brand communication and experience creates and builds a loyal base of customers that do not consider any other brand.

Creating a strong level of brand awareness takes time and strategy. Companies spend millions of dollars on marketing campaigns and advertising to increase their reach and recognition, but AI tech is able to take the guesswork out of these strategies by analyzing huge volumes of consumer data for more targeted campaigns. For example, predictive analytics software can collect, track, and analyze datasets from past customers to determine which strategies or tactics performed well. These datasets are turned into reports with insights to guide marketing efforts and place relevant content in front of the most interested eyes at the right times.

With AI-assisted marketing, advertising strategies can be backed with data to optimize ad placement. Machine learning systems can even identify the best influencers for brands to partner with in order to reach relevant audiences and grow brand familiarity.

Credit: Venturebeat.com

Consideration

The next step of the buyer’s journey is often overlooked by marketers because it can drag on for a long time, depending on the product and the customer’s needs. During the consideration phase, a customer is already familiar with a brand or product but are unsure of whether or not to actually purchase. Customers will typically research the product’s reviews, compare prices to competitors, and look for alternatives during this stage. Due to this, the number of potential customers tends to narrow down considerably as they move from this step to the decision phase.

Brands must work to combat each customer’s concerns and questions standing in the way of a purchase decision. One of the best ways to do this is by offering personalized content that is relevant to each person, making it easy for them to find the information they are seeking.

AI systems can be used to predict a customer’s needs based on consumer data and previous online behavior, and then encourage conversions with a tailored UX or even a completely customized landing page that displays content relevant to that customer.

For example, if a site visitor has viewed a certain product page and played a video demonstrating its features, these actions can trigger an AI system to target them with personalized content that prompts a conversion if they don’t proceed to buy immediately. This content could be something as simple as an email message with more information or a display ad with a special offer for the specific product.

Credit: Personyze.com

Then there are platforms that use conversational AI tech (such as chatbots and voice assistants) to power automated, text- or audio-based interactions between a business and its customers. These platforms can understand speech, decipher intent, differentiate between languages, and mimic human conversations with great accuracy. Increasingly, they are advanced enough to even understand individual context and personalize the conversation accordingly.

Based on data insights, AI tech can curate content that matches up with the issues that are most important to that person, whether it be product features, immediate delivery, long term savings, etc. Customers respond quite well to personalized offers — an Accenture study reported that 91% of consumers are more likely to purchase from a company that sent them targeted deals or recommendations.

Decision

Once a customer moves from consideration to action, AI tools can be used to support a positive sales experience and eliminate any bumps along the way. If a customer encounters an issue while browsing the site, or during checkout or payment, it could be an instant sales killer, if it isn’t handled immediately by something like live chat.

According to multiple studies, one of the most frustrating parts about online customer service is long wait times. By using AI-enabled chatbots, companies can instantly answer common questions and resolve issues or roadblocks affecting the progression of the buyer’s journey. And customers certainly appreciate these quick response times. AI systems can significantly increase conversions with effective personalization and swift customer service.

Credit: AIMultiple.com

Loyalty

The last step of the customer journey is possibly the most valuable. Over half of customers reportedly stay loyal to brands that “get them.” Returning customers also tend to spend more money than new ones, and an oft-reported stat says that on average 65% of businesses’ revenue comes from existing customers.

Businesses (and customers) can benefit greatly from loyalty programs that are backed with machine learning technology. Starbucks famously uses AI tech to analyze customer behavior, improve convenience, and identify which promotions would perform best based on that person’s drink or food preferences, location, and purchase frequency. Their loyalty program uses this data to send out thousands of offers each day for the products their customers are most likely to buy. Their customer loyalty program grew 16% YoY last year as a direct result of their Deep Brew AI engine.

Credit: Starbucks app

While a positive shopping experience and great products are certainly important factors in a customer’s decision to buy again, data-driven marketing campaigns that encourage loyalty can also help a company to grow their numbers of repeat sales. Again, AI-assisted personalization techniques can boost the chances of a customer coming back for more, especially if they receive targeted offers or shopping suggestions based on previous interactions.

Credit: Accenture.com

The Wrap

AI is proving to be the tool of the future for marketers. It allows marketing teams to use predictive insights and analytical data to encourage and assist every micro-decision taken by consumers. AI systems not only help customers move along the buyer’s journey, they can also provide a more meaningful experience along the way, leading to more conversions and brand loyalty down the road.

4 Ways Artificial Intelligence Can Impact Your Conversion Rates

At this point, there is little doubt that artificial intelligence is the future of business. The Salesforce “State of Marketing” report found that more than a fifth of businesses currently use AI for marketing purposes, including programmatic buying, personalization, and real-time offers.

At this point, there is little doubt that artificial intelligence is the future of business. The Salesforce “State of Marketing” report found that more than a fifth of businesses currently use AI for marketing purposes, including programmatic buying, personalization, and real-time offers.

artificial intelligence graphic
Credit: Salesforce

Further, AI is the fastest-growing sales technology, according to the Salesforce “State of Sales” report.

Outside of sales and marketing, companies are frequently using artificial business intelligence for tasks like reporting, dashboards, and data warehousing and analytics.

While applying AI to these business operations is certainly beneficial, it does beg the question of how exactly this technology will impact the future of conversion optimization, as well as the most important person in a business: the customer.

At the end of the day, the thing that really matters in business is the numbers. AI technology for analyst reports and predicting turns in the market is all well and good, but if it isn’t boosting sales, then what is the point?

The good news is that AI is showing promising results in terms of conversion rates, proving once again that big data is paving the way to a more profitable future for many companies. Here’s how.

1. Enriches Customer Experience

The concept of improving the customer experience (CX) is a big challenge for many reasons. CX is not merely limited to the user-friendliness of a website or the customer service that is provided; it is a combination of all of these elements. Yet another report from Salesforce found that consistency is a core element in a positive customer experience, and 70% of customers say connected processes based on earlier interactions and contextualized engagement are important for them to do business with a company.

This means that in order to improve the CX for customers, brands must adjust every part of the experience to create a coherent message.

Studies have found that customers are willing to pay more for a better experience with a business. It also has a strong effect on their likelihood to repurchase and refer the product or company to friends.

artificial intelligence graph
Credit: Temkin Group

But what exactly makes up “customer experience” and where does AI fit in?

CX is essentially the accumulation of every interaction a customer has with a business, from introduction, to purchase, to customer service. As experienced business owners know, one small kink in the journey can send people running. AI and machine learning technology can help create a more optimized experience for each customer, from start to finish.

For example, when fashion brand FlyPolar experienced a near 400% decrease in sales in the span of just four months, the business executives knew that something wasn’t right. Because most of its customers purchased online, FlyPolar used AI software to optimize its website landing pages. By using machine learning technology, this AI program “learned” which designs performed best and delivered positive results.

After several weeks of testing, the AI system identified the core roots of the conversion problems and provided the proper insights for solutions. FlyPolar created a simpler four-step conversion funnel on its website, with optimized CTA button placement throughout the landing pages. By using machine learning algorithms, FlyPolar increased its checkout page traffic by 16% and its order value by 13% in just three weeks.

This case study shows that AI technology can quickly and easily identify the root of the problem, arguably one of the most difficult parts of optimizing the CX.

The prediction capabilities of AI-powered systems can also make it easier for your customers to find exactly what they are looking for; which, in turn, improves their experience with your website. Traditional searches base results on matching keywords or similar phrases, which may or may not be accurate. In contrast, present-day search programs use ML to “learn” consumer behavior and accurately return the items that match their queries, based on their previous behavior.

ML-based search takes numerous data points into consideration, including past view and click rates, ratings, and even inventory levels to provide customers with appropriate and targeted results.

It should be no surprise here that Amazon is one of the leading retailers to utilize this kind of technology. Amazon’s recommendation engine uses item-to-item collaborative filtering to provide search results that are based on multiple data points, rather than just keyword matches. Not only does the algorithm take each customer’s past searches, purchases, and product views into consideration, but also the ratings and popularity of each item.

artificial intelligence example
Credit: Amazon

Since Amazon debuted an AI-based recommendation engine, its profits started growing exponentially. By basing search results on multiple criteria, Amazon is able to push certain products while providing shoppers with the results that fit their needs, providing a better experience for the customer with each query.

2. Enhances Personalization

Buying online is no longer a one-size-fits-all experience. In fact, customers are becoming more and more unyielding that businesses customize just about everything to fit their needs. According to Accenture’s “Personalization Pulse Check” report, three out of four customers report that they would be more likely to purchase from a brand that offers personalization and recognition than businesses that do not.

Personalization is also directly related to higher profits. Researchers have found that businesses utilizing big data systems to create personalized experiences for their customers report up to 10% higher revenues.

AI is able to take the guesswork out of personalization. One of the best examples of this strategy in action comes from Starbucks, which reported a 300% increase in customer spending thanks to its highly-customized marketing program. Customers regularly receive personalized offers and incentives to earn more points toward a free drink reward. Every customer’s offer is based on past behavior, including how often each customer purchases and which types of items the customer tends to buy.

Starbucks’ AI-powered personalization system sends out around 400,000 variants of emails with incentives that are almost entirely unique for each recipient. Due to the hyper-personalization that Starbucks offers, many customers find it easy to fulfill the requirements for these rewards. This does wonders to increase consumer participation, purchase frequency, and ultimately, customer loyalty.

artificial intelligence in loyalty programs
Credit: Starbucks App

Of course, loads of consumer data are needed in order for online companies to provide this high level of personalization. Each customer’s preferences, demographics, and behavior must be tracked and analyzed in order for brands to properly adjust their strategies to fit an individual consumer.

The results from integrating personalized messaging and marketing speak for themselves: 63% of marketers report that an increase in conversion rates was the top benefit they saw from personalization.

AI-powered personalization can be used to help customers move their way through the buyer’s journey, as well. Using ML, these programs use predictive analysis to incentivize shoppers with personalized messages, email campaigns, retargeted ads, and more.

The algorithms can study consumer behavior so that ads and other messages are sent at the right time and trigger the ideal response. For example, an algorithm that tracks customers’ click rates and scrolling habits can predict when new customers are likely to abandon their carts and send a well-timed message or personal offer to keep them engaged.

artificial intelligence-generated offer
Credit: Acquisio.com

3. Improves Results of A/B Testing

Most marketing teams and web designers rely on A/B testing to determine the best layouts, color schemes, and messaging to grab their customers’ attention. However, there are obvious limits to the “old-fashioned” testing approach. Gathering the research takes time, and there is not always a clear winner from the results.

In fact, the traditional form of this strategy may not even be effective. Jeremy Miller, marketing director at Sentient, said during an interview:

In traditional A/B testing formats, you have your control vs. an experiment. You run that experiment against your traffic, and whichever design performs better is the one you deploy … but people have found that six out of seven experiments don’t result in a positive outcome, so you actually have to put a lot of energy and resources to try to determine how you can actually increase conversions using A/B testing.”

AI can solve the three biggest problems with traditional A/B testing: time required, insight, and limited variables. By reducing these weaknesses, marketing teams have the ability to make informed design changes with the results and data to support them. Instead of taking a linear approach to testing, AI can compare thousands of variables at the same time and instantly compare the results to determine the best combination.

For example, online lingerie company Cosabella used an AI-driven testing approach when it was redesigning its website. Rather than comparing designs two at a time, like a traditional A/B test would, Cosabella was able to carry out an A/B/n experiment with 160 different design elements, simultaneously. With that many variables, it would have taken up to a year of A/B testing to gather results; with AI, the process took only seven weeks.

artificial intelligence testing
Credit: Cosabella.com

Through this testing process, Cosabella was able to determine the aesthetics that resulted in better conversions. It found that customers bought more when CTA buttons were pink, rather than black. The company also determined that family values resonated with its customers, so it did away with “free shipping” banners and replaced them with “Family Owned Since 1983.” After these short seven weeks of testing, Cosabella reported a 38% increase in conversions and a 1,000% lift in newsletter signups.

4. Speeds Up Customer Service

The faster a company can respond to customer inquiries or issues, the better. For this reason, the demand for live chat grew by 8.29% last year. Unfortunately, most businesses do not have the resources to keep their customer service departments running 24/7, leading to long response wait times for disgruntled customers.

By automating customer service with AI-powered chatbots, businesses can not only solve the issue of wait time, but also the quality of the response and assistance that customers receive.

In 2012, Amtrak’s customer service department serviced 30 million passengers each day. Obviously, with such high numbers, it was difficult to handle individual inquiries in a timely manner, so Amtrak decided to jump on the chatbot train with its AI-powered customer service rep “Julie.”

Julie was able to resolve most of these issues by pre-filling forms through scheduling tools and guiding customers step-by-step through the online booking process. Because most of these problems were handled online, the number of calls and emails decreased dramatically. At the end of the first year, Julie had answered over 5 million questions, increased booking rates by 25%, and generated 30% more revenue, thanks to upsell options included in the messaging.

artificial intelligence chat
Credit: NextIT.com

In terms of conversions, live chatbots can not only resolve issues in an instant, they can increase the chances that a customer decides to buy. When a customer’s issue is solved quickly, they are twice as likely to repurchase from that brand. Live chat is also the preferred method of communication for resolving problems or issues; however, it is important to note that the quality of the messaging far outweighs the speed of the response.

According to Kayako’s report on live chat service, 95% of customers say that receiving a thorough response that answers their question or resolves the problem is more important than just getting a quick reply. This is a major issue that many companies have with AI chatbots; they are simply programmed to give automated, scripted responses, which 29% of customers report as simply frustrating and unhelpful.

This is where AI-based chatbots save the day; they can adjust their messaging based on FAQs, as well as the customer’s phrasing and responses. This process leads to better and more natural replies from bots that delight customers and give them the timely information they need.

An AI chatbot is not a one-time fix to the issue of customer service. It is a strategy that must be properly monitored, adjusted, and perfected over time in order to deliver the best results.

The Wrap

Many conversations these days are revolving around AI and its impact on the future of business. And, quite honestly, it seems like the answer to just about every current business planning issue out there. Predictive analytics can tell you when things are about to change. Machine learning can understand your customers on a personal, granular level, and big data can keep track of every metric for accurate reporting.

However, one of the clearest benefits of AI is the direct impact it can have on conversions. It eliminates the guesswork from improving the CX of webpages and delivers timely and accurate testing results needed to increase the likeliness of conversions. Big data systems and AI make hyper-personalization possible to customize the experience for each visitor. Finally, chatbots can use ML to instantly engage with customers, resolve issues immediately, and close sales.

Success all boils down to how a business makes the customer feel. Most of the time, this is what determines whether or not a customer will purchase. Studies have found, unsurprisingly, that when customers feel special, important, and satisfied, they are more likely to buy from those brands. AI gives brands the power to do just that.

Stop Expecting Data Scientists to Be Magical: Analytics Is a Team Sport

Many organizations put unreasonable expectations on data scientists. Their job descriptions and requirements are often at a super-human level. “They” say — and who are they? — that modern-day data scientists must be good at absolutely everything. Okay, then, what’s “everything,” in this case?

Many organizations put unreasonable expectations on data scientists. Their job descriptions and requirements are often at a super-human level. “They” say — and who are they? — that modern-day data scientists must be good at absolutely everything. Okay, then, what’s “everything,” in this case?

First, data scientists have to have a deep understanding in mathematics and statistics, covering regression models, machine learning, decision trees, clustering, forecasting, optimization, etc. Basically, if you don’t have a post-graduate degree in statistics, you will fail at “hello.” The really bad news is that even people with statistics degrees are not well-versed in every technique and subject matter. They all have their specialties, like medical doctors.

Then data scientists have to have advanced programming skills and deep knowledge in database technologies. They must be fluent in multiple computer languages in any setting, easily handling all types of structured and unstructured databases and files in any condition. This alone is a full-time job, requiring expert-level experience, as most databases are NOT in analytics-ready form. It is routinely quoted that most data scientists spend over 80% of their time fixing the data. I am certain that these folks didn’t get an advanced degree in statistics to do data plumbing and hygiene work all of the time. But that is how it is, as they won’t see what we call a “perfect” dataset outside schools.

Data scientists also have to have excellent communication and data visualization skills, being able to explain complex ideas in plain English. It is hard enough to derive useful insights out of mounds of data; now they have to construct interesting stories out of them, filled with exciting punchlines and actionable recommendations at the end. Because most mortals don’t understand technical texts and numbers very well — many don’t even try, and some openly say they don’t want to think — data scientists must develop eye-popping charts and graphs, as well, using the popular visualization tool du jour. (Whatever that tool is, they’d better learn it fast).

Finally, to construct the “right” data strategies and solutions for the business in question, the data scientist should have really deep domain and industry knowledge, at a level of a management and/or marketing consultant. On top of all of that, most job requirements also mention soft skills — as “they” don’t want some data geeks with nerdy attitudes. In other words, data scientists must come with kind and gentle bedside manners, while being passionate about the business and boring stuff like mathematics. Some even ask for child-like curiosity and ability to learn things extremely fast. At the same time, they must carry authority like a professor, being able to influence non-believers and evangelize the mind-numbing subject of analytics. This last part about business acumen, by the way, is the single-most important factor that divides excellent data scientists who add value every time they touch data, and data plumbers who just move data around all day long. It is all about being able to give the right type of homework to themselves.

Now, let me ask you: Do you know anyone like this, having all of these skills and qualities in “one” body? If you do, how many of them do you personally know? I am asking this question in the sincerest manner (though I am quite sarcastic, by nature), as I keep hearing that we need tens of thousands of such data scientists, right now.

There are musicians who can write music and lyrics, determine the musical direction as a producer, arrange the music, play all necessary instruments, sing the song, record, mix and master it, publish it, and promote the product, all by themselves. It is not impossible to find such talents. But if you insist that only such geniuses can enter the field of music, there won’t be much music to listen to. The data business is the same way.

So, how do we divide the task up? I have been using this three-way division of labor — as created by my predecessors — for a long time, as it has been working very well in any circumstance:

  • A Statistical Analyst will have deep knowledge in statistical modeling and machine learning. They would be at the core of what we casually call analytics, which goes way beyond some rule-based decision-making. But these smart people need help.
  • A Master Data Manipulator will have excellent coding skills. These folks will provide analytics-ready datasets on silver platters for the analysts. They will essentially take care of all of the “before” and “after” steps around statistical modeling and other advanced analytics. It is important to remember that most projects go wrong in data preparation and post-analytics application stages.
  • A Business Analyst will need to have a deep understanding of business challenges and the industry landscape, as well as functional knowledge in modeling and database technologies. These are the folks who will prescribe solutions to business challenges, create tangible projects out of vague requests, evaluate data sources and data quality, develop model specifications, apply the results to businesses, and present all of this in the form of stories, reports, and data visualization.

Now, achieving master-level expertise in one of these areas is really difficult. People who are great in two of these three areas are indeed rare, and they will already have “chief” or “head” titles somewhere, or have their own analytics practices. If you insist only procuring data scientists who are great at everything? Good luck to you.

Too many organizations that are trying to jump onto this data bandwagon hire just one or two data scientists, dump all kinds of unorganized and unstructured data on them, and ask them to produce something of value, all on their own. Figuring out what type of data or analytics activity will bring monetary value to the organization isn’t a simple task. Many math geeks won’t be able to jump that first hurdle by themselves. Most business goals are not in the form of logical expressions, and the majority of data they will encounter in that analytics journey won’t be ready for analytics, either.

Then again, strategic consultants who develop a data and analytics roadmap may not be well-versed in actual modeling, machine learning implementation, or database constructs. But such strategists should operate on a different plane, by design. Evaluating them based on coding or math skills would be like judging an architect based on his handling of building materials. Should they be aware of values and limitations of data-related technologies and toolsets? Absolutely. But that is not the same as being hands-on, at a professional level, in every area.

Analytics has always been a team sport. It was like that when the datasets were smaller and the computers were much slower, and it is like that when databases are indeed huge and computing speed is lightning fast. What remains constant is that, in data play, someone must see through the business goals and data assets around them to find the best way to create business value. In executing such plans, they will inevitably encounter many technical challenges and, of course, they will need expert-level technicians to plow through data firsthand.

Like any creative work, such as music producing or movie-making, data and analytics work must start with a vision, tangible business goals, and project specifications. If these elements are misaligned, no amount of mathematical genius will save the day. Even the best rifles will be useless if the target is hung in a wrong place.

Technical aspects of the work matter only when all stakeholders share the idea of what the project is all about. Simple statements like “maximizing the customer value” need a translation by a person who knows both business and technology, as the value can be expressed in dollars, visits, transactions, dates, intervals, status, and any combination of these variables. These seemingly simple decisions must be methodically made with a clear purpose, as a few wrong assumptions by the analyst at-hand — who may have never met the end-user — can easily derail the project toward a wrong direction.

Yes, there are people who can absolutely see through everything and singlehandedly take care of them all. But if your business plan requires such superheroes and nothing but such people, you must first examine your team development roadmap, org chart, and job descriptions. Keep on pushing those poor and unfortunate recruiters who must find unicorns within your budget won’t get you anywhere; that is not how you’re supposed to play this data game in the first place.

Even AI Needs Clean Data in Order to Be the Shiny Object

Users are quickly realizing that investing in AI is not the end of the road. Then again, in this analytics journey, there really is no end anyway; much like the scientific journey, it is a constant series of hypothesis, testing, and course corrections.

Users are quickly realizing that investing in AI is not the end of the road. Then again, in this analytics journey, there really is no end anyway; much like the scientific journey, it is a constant series of hypothesis, testing, and course corrections. And now, I’ll explain why that means even AI needs clean data.

If there is a book out there — many have asked me about it — it would look more like a long series of case studies, not some definitive roadmap for all. Why? Because prescribing analytics is much like a doctor’s work. It depends as much on the unique situation of the patient as on the list of solutions.

That is the main reason why one cannot just install AI and call it a day. Who’d give it a purpose, guide it, and constantly fine-tune it? Not itself, for sure.

Then there is a question about what goes into it. AI — or any type of analytics tool, for that matter — depends on clean and error-free data. If the data are dirty and unusable, you may end up automating inadequate decision-making processes, getting wrong answers really fast. I’d say that would be worse than not having any answer at all.

So far, you may say I am just stating the obvious here. Of course, AI or machine learning require clean and error free data. The real trouble is that such data preparation often takes up as much as 80% (if not more) of the whole process of applying data-based intelligence to decision-making. In fact, users are finding out that the algorithmic part of the equation is the simplest to automate. The data refinement process is far more complicated than that, as it really depends on the shape of the available data. And some are really messy (hence, the title of my series in this fine publication, “Big Data, Small Data, Clean Data, Messy Data”).

So, why aren’t data readily usable?

  • Data Are in Silos: This is so common that “siloed data” is actually a term that we commonly use in meeting rooms. Simply, if the data are locked up somewhere, they won’t be much of use for anyone. Worse, each silo may be on a unique platform, with incompatible data formats from others.
  • Data Are in One Place, But Not Connected: Putting the data in one place isn’t enough, if they are not properly connected. Let’s say an organization is pursuing the coveted “Customer 360” (or more properly, “360-degree view of a customer”) for personalized marketing. The first thing to do is to define what a “person” means, in the eyes of the machine and algorithms. It could be any form of PII or even biometrics data, through which all related data would be merged and consolidated. If the online and offline shopping history of a person aren’t connected properly, algorithms will treat them as two separate entities, devaluating the target customer. This is just one example; all kinds of analytics — whether they be forecasting, segmentation, or product analysis — perform better with more than one type of data, and they should be in one place to be useful.
  • Data Are Connected, But Many Fields Are Wrong or Empty: So what if the data are merged in one place? If data are mostly empty or incorrect, they will be worse than not having any at all. Good luck forecasting or predicting anything with data fields with really low fill rates. Unfortunately, we encounter tons of missing values in the case of “Customer 360.” What we call Big Data have lots of holes in them, when everything is lined up around the target (i.e., it is nearly impossible to know everything about everyone). Plus, remember that most modern databases record and maintain what are available; but in predictive analytics, what we don’t know is equally important.
  • Data Are There, But They Are Not Readily Usable, as They Are in Free-Form Formats: You may have the data, but they may need some serious standardization, refinement, categorization, and transformation processes to be useful. Many times I encountered hundreds, at time over a thousand, offer and promotion codes. To find out “what marketing efforts worked,” we would have to go through some serious data categorization to make them useful. (Refer to “The Art of Data Categorization”) This is just one example of many. Too often, analytics work is stuck in the middle of too much free-form, unstructured data.
  • Data Are Usable, But They Are One-Dimensional: Bits and pieces of data, even if they are clean and accurate, do not provide a holistic portrait of target individuals (if the job is about 1:1 marketing). Most predictive analytics work requires diverse data of a different nature, and only after proper data consolidation and summarization, we can obtain a multi-dimensional view. So-called relational databases and unstructured databases do not provide such a perspective without data summarization (or de-normalization) processes, as entities of such databases are just lists of events and transactions (e.g., on such and such date, this individual clicked some email link and bought a particular item for how much).
  • Data Are Cleaned, Consolidated, and Summarized, But There Is No Built-in Intelligence: To predict what the target individual is interested in, data players must rearrange the data to describe the person, not just events or transactions. Why do you think even large retailers, like Amazon, treat you like you are only about the very last transaction, sending the “likes” of the last item you purchased, ignoring years of interaction history? Because their data are not describing “you” as a target. And you are not just a sum of past transactions, either. For instance, your days in between purchases in the home electronics category may be far greater than those in the apparel category, yet showing higher average spending in the first category. This type of insight only comes out when the data are summarized properly to describe the buyer, not each transaction. Further, summarized data should be in the form of answers to questions, acting as building blocks of predictive scores. Intelligent variables always increase the predictive power of models, machine-based or not.
  • Data Variables Include Intelligence, But It Is Still Difficult to Derive Insights: Lists of intelligent variables are just basic necessities for advanced analytics, which would lead us to deeper and actionable insights. Even statisticians and analysts require a long training period to derive meanings out of seemingly beautiful charts and effectively develop stories around them. Yes, we can see that certain product sales went down, even with heavy promotion. But what does that really mean, and what should we do about it? For a machine to catch up with that level of storytelling, the data best be on silver platters in pristine condition first. Because changing assumptions based on “what is not there” or “what looks suspicious” is still in the realm of human intuition. Machines, for now, will read the results as if every bit of input data is correct and carries equal weight.

There are schools of thought that machines should be able to take raw data in any form, and somehow spit out answers for us mortals. But I do not subscribe to such a brute-force approach. Even if there is no human intervention in the data refinement process, machines will have to clean data in steps, like we have been doing. Simply put, a machine that is really good at identifying target individuals will be separately trained from the one that is designed for prediction of any kind.

So, what does clean and useful data mean? Just reverse the list above. In summary, good data must be:

  • Free from silos
  • Properly connected, if coming from disparate sources
  • Free from errors and too many missing values (i.e., must have good coverage)
  • Readily usable by non-specialists without having to manipulate them extensively
  • Multi-dimensional as a result of proper data summarization
  • In forms of variables with built-in intelligence
  • Presented in ways that provide insights, beyond a simple list of data points

Then, what are the steps of data refinement process? Again, if I may summarize the key steps out of the list above:

  1. Data collection (from various sources)
  2. Data consolidation (around the key object, such as individual target)
  3. Data hygiene and standardization
  4. Data categorization
  5. Data summarization
  6. Creation of intelligent variables
  7. Data visualization and/or modeling for business insights

Conclusion

I have covered all of these steps in detail through this column over the years. Nevertheless, I just wanted to share these steps on a high level again, as the list will serve as a checklist, of sorts. Why? Because I see too many organizations — even the advanced ones — that miss the whole category of necessary activities. How many times have I seen unstructured and uncategorized data, and how many times have I seen very clean data but only on an event and transaction level? How can anyone predict the target individual’s future behavior that way, with or without the help of machines?

The No. 1 reason why AI or machine learning do not reach their full potential is inadequate input data. Imagine putting unrefined oil as fuel or lubricant for a brand new Porsche. If the engine stalls, is that the car’s fault? To that point, please remember that even the machines require clean and organized data. And if you are about to have machines do the clean-up, also remember that machines are not that smart (yet), and they work better when trained for a specific task, such as pattern recognition (for data categorization).

One last parting thought: I am not at all saying that one must wait for a perfect set of data. Such a day will never come. Errors are inevitable, and some data will be missing. There will be all kinds of collection problems, and the limitation in data collection mechanisms cannot be fully overcome, thanks to those annoying humans who don’t comply well with the system. Or, it could be that the target individual simply did not create an event for the category yet (i.e., data will be missing for the Home Electrics category, if the buyer in question simply did not do anything in that category).

So, collect and clean the data as much as possible, but don’t pursuit 100% either. Analytics — with or without machines — always have been making the most of what we have. Leave it at “good enough,” though machine wouldn’t understand what that means.

Machine Learning? I Don’t Think Those Words Mean What You Think They Mean

I find more and more people use the term “machine learning” when they really mean to say “modeling.” I guess that is like calling all types of data activities — with big and small data — “Big Data.” And that’s OK.

I find more and more people use the term “machine learning” when they really mean to say “modeling.” I guess that is like calling all types of data activities — with big and small data — “Big Data.” And that’s OK.

Languages are developed to communicate with other human beings more effectively. If most people use the term to include broader meanings than the myopic definition of the words in question, and if there is no trouble understanding each other that way, who cares? I’m not here to defend the purity of the meaning, but to monetize big or small data assets.

The term “Big Data” is not even a thing in most organizations with ample amounts of data anymore, but there are many exceptions, too. I visit other countries for data and analytics consulting, and those two words still work like “open sesame” to some boardrooms. Why would I blame words for having multiple meanings? The English dictionary is filled with such colloquial examples.

I recently learned that famous magic words “Hocus Pocus” came from the Latin phrase “hoc est corpus,” which means “This is the body (of Christ)” as spoken during Holy Communion in Roman Catholic Churches. So much for the olden-day priests only speaking in Latin to sound holier; ordinary people understood the process as magic — turning a piece of bread into the body of Christ — and started applying the phrase to all kinds of magic tricks.

However, if such transformations of words start causing confusion, we all need to be more specific. Especially when the words are about specific technical procedures (not magic). Going back to my opening statement, what does “machine learning” mean to you?

  • If spoken among data scientists, I guess that could mean a very specific way to describe modeling techniques that include Supervised Learning, Unsupervised Learning, Reinforced Learning, Deep Learning, or any other types of Neural Net modeling, indicating specific methods to construct models that serve predetermined purposes.
  • If used by decision-makers, I think it could mean that the speaker wants minimal involvement of data scientists or modelers in the end, and automate the model development process as much as possible. As in “Let’s set up Machine Learning to classify all the inbound calls into manageable categories of inquiries,” for instance. In that case, the key point would be “automation.”
  • If used by marketing or sales; well, now, we are talking about really broad set of meanings. It could mean that the buyers of the service will require minimal human intervention to achieve goals. That the buyer doesn’t even have to think too much (as the toolset would just work). Or, it could mean that it will run faster than existing ways of modeling (or pattern recognition). Or, they meant to say “modeling,” but they somehow thought that it sounded antiquated. Or, it could just mean that “I don’t even know why I said Machine Learning, but I said it because everyone else is saying it” (refer to “Why Buzzwords Suck”).

I recently interviewed a candidate fresh out of a PhD program for a data scientist position, whose resume is filled with “Machine Learning.” But when we dug a little deeper into actual projects he finished for school work or internship programs, I found out that most of his models were indeed good, old regression models. So I asked why he substituted words like that, and his answer was staggering; he said his graduate school guided him that way.

Why Marketers Need to Know What Words Mean

Now, I’m not even sure whom to blame in a situation like this, where even academia has fallen under the weight of buzzwords. After all, the schools are just trying to help their students getting high paying jobs before the summer is over. I guess then the blame is on the hiring managers who are trying to recruit candidates based on buzzwords, not necessarily knowing what they should look for in the candidates.

And that is a big problem. This is why even non-technical people must understand basic meanings of technical terms that they are using; especially when they are hiring employees or procuring outsourcing vendors to perform specific tasks. Otherwise, some poor souls would spend countless hours to finish things that don’t mean anything for the bottom-line. In a capitalistic economy, we play with data for only two reasons:

  1. to increase revenue, or
  2. to reduce cost.

If it’s all the same for the bottom line, why should a non-technician care about the “how the job is done” part?

Why It Sucks When Marketers Demand What They Don’t Understand

I’ve been saying that marketers or decision-makers should not be bad patients. Bad patients won’t listen to doctors; and further, they will actually command doctors prescribe certain medications without testing or validation. I guess that is one way to kill themselves, but what about the poor, unfortunate doctor?

We see that in the data and analytics business all of the time. I met a client who just wanted to have our team build neural net models for him. Why? Why not insist on a random forest method? I think he thought that “neural net” sounded cool. But when I heard his “business” problems out, he definitely needed something different as a solution. He didn’t have the data infrastructure to support any automated solutions; he wanted to know what went on in the modeling process (neural net models are black boxes, by definition), he didn’t have enough data to implement such things at the beginning stage, and projected gains (by employing models) wouldn’t cover the cost of such implementation for the first couple of years.

What he needed was a short-term proof of concept, where data structure must be changed to be more “analytics-ready.” (It was far from it.) And the models should be built by human analysts, so that everyone would learn more about the data and methodology along the way.

Imagine a junior analyst fresh out of school, whose resume is filled with buzzwords, meeting with a client like that. He wouldn’t fight back, but would take the order verbatim and build neural net models, whether they helped in achieving the business goals or not. Then the procurer of the service would still be blaming the concept of machine learning itself. Because bad patients will never blame themselves.

Even advanced data scientists sometimes lose the battle with clients who insist on implementing Machine Learning when the solution is something else. And such clients are generally the ones who want to know every little detail, including how the models are constructed. I’ve seen data scientists who’d implemented machine learning algorithms (for practical reasons, such as automation and speed gain), and reverse-engineered the models, using traditional regression techniques, only to showcase what variables were driving the results.

One can say that such is the virtue of a senior-level data scientist. But then what if the analyst is very green? Actually some decision-makers may like that, as a more junior-level person won’t fight back too hard. Only after a project goes south, those “order takers” will be blamed (as in “those analysts didn’t know what they were doing”).

Conclusion

Data and analytics businesses will continually evolve, but the math and the human factors won’t change much. What will change, however, is that we will have fewer and fewer middlemen between the decision-makers (who are not necessarily well-versed in data and analytics) and human analysts or machines (who are not necessarily well-versed in sales or marketing). And it will all be in the name of automation, or more specifically, Machine Learning or AI.

In that future, the person who orders the machine around — ready or not — will be responsible for bad results and ineffective implementations. That means, everyone needs to be more logical. Maybe not as much as a Vulcan, but somewhere between a hardcore coder and a touchy-feely marketer. And they must be more aware of capabilities and limitations of technologies and techniques; and, more importantly, they should not blindly trust machine-based solutions.

The scary part is that those who say things like “Just automate the whole thing with AI, somehow” will be the first in line to be replaced by the machines. That future is not far away.

Automation — With a Little Help From Good Machines

Some claim that human behaviors are just algorithmic responses developed over past 70,000 years or so. Now, armed with data that we are casually scattering around, machine-based algorithms outperform human brains in most areas already, and such evolution will continue.

We should be mindful when dropping buzzwords (refer to “Why Buzzwords Suck”). As more and more people jump on the bandwagon of a buzzword, it tends to gain magical power. Eventually, some may even believe that buying into a “word” will solve all their problems.

But does it ever work out that way? Did anyone make a fortune buying into the Big Data hype yet? I know some companies did; but, ironically, the winners do not even utter such words. I’ve never seen any news release from Google or Amazon that they are investing in “Big Data.” For them, playing with large amounts of data have been just part of their businesses all along.

Now the new buzz is about AI, machine learning and automation, in general; and it will be a little different from buzzwords from the past. Whether we like it or not, that is the direction that we are already headed in the world where each decision will be increasingly more dependent on deterministic algorithms.

Some even claim that human behaviors are just algorithmic responses developed over past 70,000 years or so. Now, armed with data that we are casually scattering around, machine-based algorithms outperform human brains in most areas already, and such evolution will continue until most humans will become largely irreverent in terms of economic value, they say. Not that it would happen overnight, but the next generation may look at our archaic way of things the way we look at our ancestors who were without computers.

First, the Marketing Case for AI

If such is our fate, why are contemporary humans so willingly jumping onto this automation bandwagon where machines will make decisions for us? Because they are smarter than average humans? What does “smart” even mean when we are talking about machines? I think people generally mean to say that machines remember details better than us, and calculate a complex series of algorithms faster and more accurately than us.

Some may say that humans with experiences are wiser with visions to see through things that are not seemingly related. But I dare to say that I’ve seen machines from decades ago finding patterns that humans would never find on their own. When machines start learning without our coaching or supervision — the very definition of AI — at a continuously increasing rate, no, we won’t be able to claim that we are wiser than machines, either. In the near future, if not already.

So, before we casually say that AI-based automation is the future of marketing, let’s ask ourselves why we are so eager to give more power to machines. For what purpose?

The answer to that philosophical question in the business world is rather simple; decision-makers are jumping onto the automation bandwagon to save money. Period.

Specifically, by reducing the number of people who perform tasks that machines can do. As a bonus, AI saves time by performing the tasks faster than ever. In some cases — mostly, for small operations — machines will perform duties that have been neglected due to high labor costs, but even in such situations, automation will not be considered a job-creating force.

Making the Marketing Case for Humans Using Data

Some may ask why I am stating the obvious here. My intention here is to emphasize that automation, all by itself, doesn’t have the magic power to reveal new secrets, as the technology is primarily a replacement option for human labor. If the result of machine-based analytics look new to you, it’s because humans in your organization never looked at the data the same way before, not because it was an impossible task to begin with. And that is a good thing as, in that case, we may be talking about using machine power to do the things that you never had human resources for. But in most cases, automation is about automating things that people know how to do already in the name of time and cost savings.

Like any other data or analytics endeavors, we must embark on marketing automation projects with clear purposes. What would be the expected outcome? What are you trying to achieve? For what types of tasks? What parts of the process are we automating? In what sequence?

Just remember that anyone who would say “just automate everything” is the type of person who would be replaced by machines first.

At the end of that automation rainbow, there lie far less people employed for given tasks, and only the logical ones who see through the machines would remain relevant in the new world.

Nonetheless, providing purposes for machines is still a uniquely human function, for now. And project goals would look like those of any other tasks, if we come back to the world of marketing here. Examples are:

  • Consolidate unorganized free-form data into intelligent information — for further analyses, or for “more” automation of related tasks. For instance, there are thousands of reasons why consumers call customer service lines. Machines can categorically sort those inquiries out, so that finding proper answers to them — the very next logical step — can also be automated. Or, at least make the job easier for the operator on the call (for now). Deciphering image files would be another example, as there has been no serious effort to classify them with sheer manpower in a large scale. But then again, is it really impossible for humans to classify large numbers of images? How about crowdsourcing? Or let an authoritarian government force a stadium-full of North Koreans to do it manually? We’d use machines, because it would be just cheaper and faster to do it with machine learning. But who do you think corrected wrong categorization done by machines to make them better?
  • Find the next, best product for the buyer. This one is quite a popular task for machines, but even a simple “If you bought this, you would like that, too” type of product recommendation would work far better if input data (i.e., product descriptions and product categories) were well-organized — by machines. Machines work better in steps, too.
  • Predict responsiveness to channel promotions and future value of a customer. These are age-old tasks for analytics teams, but with sets of usable data, machines can update algorithms and apply scores, real-time, as new information enters the system. Call that AI, if algorithms are updated automatically, all on its own. Actually, this would be easier for a machine to pick up than fixing messy data. Not that they will know the difference between easy and difficult, but I’m talking about in terms of ease of delegation, from our point of view.
  • Then ultimately, personalize every interaction with every customer through every touch channel. I guess that would be the new frontier for marketers, as approaching personalization on such massive scale can’t be done without some help from good machines. But I still stand by my argument that each component of personalization efforts is something that we know how to do (refer to “Key Elements of Complete Personalization”). By performing each step much faster with machines, though, we can soon reach that ultimate level of personalization through consolidation of services and tasks. And the grand design of such a process will be set up by humans — at least initially.

This Human’s Final Thoughts on AI

These are just some examples in marketing.

If we dive into the operational side, there will be an even richer list of candidates for automation.

In any case, how do marketers stay a step ahead of machines, and remain commanders of them?

Ironically, we must be as logical as a Vulcan to control them effectively. Machines do not understand illogical commands, and will ignore them without any prejudice (but it would “feel” like disrespect to us).

Teaching Humans to Automate

I heard that some overzealous parents started teaching computer programming to 4- or 5-year old children, in addition to a foreign language and piano lessons. That sounds all Cool and the Gang to me, but I wondered how they would teach such young kids how to code.

Obviously, they wouldn’t teach them JavaScript or Python from Day 1. Instead, they first teach the kids how to break down simple tasks into smaller steps. For example, if I ask you to make a grilled cheese sandwich, you — as a human being — will go at it with minimal instruction. Try to order an imaginary machine to do the same. For the machine’s sake, it won’t even know what a grilled cheese sandwich is, or understand why carbon-based lifeforms (especially gluttonous humans) must consume such large quantities of organic materials on a regular basis.

Teaching Machines to Human

If you try it, you will find that the task of writing a spec for a machine is surprisingly tedious.

Just for a little grilled cheese sandwich, you have to:

human automation, the grilled cheese story
Photo by: Christoher Del Rosario (www.christopherdelrosario.com) | Credit: Getty Images by Christopher Del Rosario / EyeEm
  • instruct it on how to get to the breadbox,
  • how to open it,
  • how many slices of bread should be taken out,
  • how to take them out without flattening them (applying the right amount of pressure),
  • how to open the refrigerator,
  • how to locate butter and cheese in the mix of many food items,
  • how to peel off two slices of cheese without tearing them,
  • how to ignite a stove burner,
  • how to find a suitable pan (try to explain “suitable,” in terms measurements and shape),
  • how to preheat the pan to a designated temperature (who’d design and develop the heat censor?),
  • how to melt butter on the pan without burning it,
  • how to constantly measure and monitor the temperature,
  • how to judge the right degree of “brown” color of grilled cheese,
  • etc. etc..

If you feel sick reading all of this, well, I didn’t even get to the part about serving the damn sandwich on a nice plate yet.

Anyway, Human Marketers, Here’s the Conclusion

I am not at all saying that all decision-makers must be coders. What I am trying to emphasize is the importance of breaking down a large task into smaller “logical” steps. Smart machines will not need all of these details to perform “known” tasks (i.e., someone else taught it already). And that is how they get smarter. But they would still work better in clear logical steps.

For humans to command machines effectively, we must think like machines — at least a little bit. Yes, automation is mostly about automating things we already know how to do. We use machines to perform those tasks much faster than humans. To achieve overall organizational effectiveness, break down the processes into smaller bits, where each step becomes the stepping stones for the next. Then prioritize which part would be the best candidate for automation, and which part would still be best served by human brains and hands.

For now, that would be the fastest route to full automation. As a result of it, many humans may be demoted to jobs like reading machine-made scripts to other humans on the phone, or delivering items that machines picked for human consumers in the name of personalization. If that is the direction where human collectives are headed, let’s try to be the ones who provide purposes for machines. Until they don’t even need such instructions from us anymore.

Why You’re Probably Not Ready for Machine Learning, and How to Get There

The truth is, there’s a gap there. Companies have spent years developing data capabilities to support the various flavors of direct, automated and digital marketing. Now that effort needs to shift toward corralling even larger data sets for even more business-benefit in machine learning.

Last week on our Marketing Garage podcast, we interviewed a Fortune 500 company, Reckitt Benkiser, about the success they’ve had using machine learning to guide their marketing strategy across a number of well-known consumer brands.

As we went through the interview, I couldn’t help but wonder why more brands aren’t using machine learning in that way. Then at FUSE Digital Marketing, I got to hear Christopher Penn, co-founder of Trust Insights and host of the Marketing Over Coffee podcast, talk about where marketing data is, and where it needs to be for machine learning to add value.

The truth is, there’s a gap there. Companies have spent years developing data capabilities to support the various flavors of direct, automated and digital marketing. Now that effort needs to shift toward corralling even larger data sets for even more business benefit.

That’s why Reckitt Benkiser has both their own internal head of e-commerce analytics and still brings in help from Fractal Analytics to tap into AI insights. And why more marketers need to lay the data ground work to make machine learning work for them in the future.

Artificial Intelligence Is Math, Not Magic

“All machine learning begins with stats and probability,” said Penn. “Machine learning is math, it’s not magic.”

And when it comes to applying that math to today’s marketing data, he identified five core problems:

  1. Volume: There’s so much data being crated today that any attempt to quantify it just sounds like you’re making up words. For example, Penn said “this year it is estimated, as a civilization, we’re going to create about 30 zettabytes of data.” Your brain cannot count to a “zettabyte.” (Go ahead and try, I’ll wait.)
  2. Variety: There are more kinds of data than ever.
  3. Velocity: It’s coming at us faster than ever.
  4. Veracity: Data has to be verified. If it’s not accurate, it’s harmful to your business.
  5. Value: Penn compared data to oil. Raw, it’s sticky and not very useful. It has to be refined and used to create value.

Machine learning can help marketers overcome all of those problems, but you need to get your data and organization into position to process it and put it to work.

One note: There is a difference between artificial intelligence and machine learning. According to Penn, AI is when you develop an algorithm that allows a computer to some extent to “think” for you towards achieving a goal. Machine learning is when you let the computer create an algorithm to solve for the goals you give it based on large pools of data. (And “deep learning” is when you have many machine learning algorithms with deep pools of data working in conjunction.) They’re sort of different levels of the same idea.

The Enterprise AI Journey

Penn says there are five stages of enterprise data usage.

  1. Identification: You can use data to identify what happened.
  2. Diagnostic: You can use data to diagnose problems and why it happened.
  3. Predictive: You can predict what will happen.
  4. Prescriptive: You can use data to determine what should happen.
  5. Proactive: A machine can use data to make it happen for you.

Most companies are somewhere around stages 2 and 3. Getting to that fifth stage is an enterprise AI journey of establishing the capabilities to move form identifying data to proactively using it. Penn sees that as a seven-step process:

  1. Data Foundation: You have the base tools to house and process data.
  2. Measurement and Analytics: You are able to accurately measure and analyze what you’re doing.
  3. Insights & Research: You have the tools to turn the measurement and analytics into business intelligence.
  4. Process Automation: You have the tools to automate these processes, so turning your data foundation into business intelligence happens automatically with minimal personnel intervention.
  5. Data Science: You install the capability to recognize next-level data science insights.
  6. Machine Learning: You have the data science  and tools in place to integrate AI insights into your business.
  7. AI Powered Enterprise: You solve for AI first, breaking down business challenges into process-oriented steps that the AI can solve with minimal human intervention. “How can we use AI to do this for us on an iterative, continuously optimizing process o create business value?”

Again, almost no one has gotten to step 7, or even 6. Th most advanced brands today around around step 5, and most are still in the first half of the journey.

Should You Buy or Build AI?

If you feel like your data and processes are ready for AI, Penn says whether to buy or build yourself comes down to time, money and strategy.

If you have money but no time, a vendor can help. Look to buy or hire a service.

If you have time but not money, look to build in-house. A lot of AI in use today is using open source software. There’s no fee, just a knowledge cost. Budget about two years per person to get up to speed. (For many marketers in 2018, this could be prohibitive.)

Strategy: Before doing either, make sure you’re going to be able to use what you get to add value to the company. Bot of those options are significant investments.  Do you have the strategy to support AI whichever you choose? One good benchmark is to look at the state of your digital transformation. A complete digital transformation is essential to do this. Without being digital, you can’t be AI.

How to Prepare Your Company for AI

Once you’re ready and committed, Penn said you need three kinds of people you need to have to enable your AI strategy.

To go back to the oil analogy, data is messy stuff that’s useless on its own without refinement. So you need:

  • Developers to extract the data
  • Data scientists to refine the data
  • Marketing technologies to figure out how to use that data

Beyond those roles, certain skills become more important in a digital, AI-driven business. So look to train or hire overall talent with skills like these:

A slide from FUSE Digital Marketing 2018 where Christopher Penn laid out the essential skill to thrive in a machine learning world.
A slide from FUSE Digital Marketing 2018 where Christopher Penn laid out the essential skill to thrive in a machine learning world. | Credit: FUSE Digital Marketing 2018 by Christopher Penn

It also takes some different ways of thinking, according to Penn.

To be successful in an AI enterprise, marketers must learn to think algorithmically. Learn to think like a machine, process-focused on how to solve that problem.

Also, it’s important to understand that machines need oversight. Algorithms can spit out a bias if you don’t actively watch for it. It happened to Amazon, it’s happened in police use, and it can happen to you to if you don’t watch.

And finally, AI-focused organization must be built around outcome-focused people. It’s easy to get lost in AI if your goals aren’t clear, measurable and accountable.

Who Does the Machine Serve?

In the future, Penn said there will be two kinds of jobs: “Either you will manage the machines, or the machines will manage you. We want to be the people who manage the machines, not the kind of people who are managed by them.”

Video Q&A: How Will AI Help Marketers Improve Retargeting & Conversion?

There’s a lot of loose talk around the potential for AI to change the nature of the marketing game, but beyond the buzz it can be hard to tell exactly how marketers will be using it to improve their businesses. In a series of video Q&A’s, marketing AI practitioner and Trust Insights co-founder Christopher Penn will explain how marketers can actually use AI.

Penn will be leading the keynote session on AI applications in marketing at the FUSE Digital Marketing Summit this November. Learn more about attending here.

Check out Penn’s previous videos:
Q: How Will AI Help Marketers Tap Their Data Wells?
Q: What Marketing Processes or Tasks Will AI Eliminate?

Q: How will AI help marketers with retargeting and sales conversion?

This is a really interesting question because one of the things that marketers struggle with is what causes a conversion. What factors, what measures, what metrics, what learners, what dimensions lead to conversion or contribute to conversion? A big part of this is the foundation of attribution analysis. What pieces of data have driven conversions in the past? And then, with things like retargeting, you’re trying to focus on predicting what things are likely to cause conversions in the future. The way AI and machine learning help with this is dealing with what are called “weak learners.”

A weak learner is any dimension or metric whose predictive power is just barely above random chance. It’s called a weak learner because it’s a weak signal. It’s not a signal that by itself is a very strong signal. So for example, the number of times someone has retweeted your tweets, right? For a fair number of businesses, that’s going to be a really weak learner. In fact, it may or may not even be statistically relevant. But at the very least that is probably going to be a weak learner.

There are also things like how many times someone has opened an email, the number of social channels someone follows you on, the pages they visited on a website, the amount of time they spent on a page.

When you think about all the data that we have access to as marketers and then we consider that most of these metrics are pretty weak, you get to start getting a sense of the scope of the problem.

We have all this data and none of it is the one answer that we’re looking for. The answer that says, “This is the thing we need to do more of.” It would be nice if it didn’t work that way. It would be nice to know you should always send email on Tuesdays, that’s going to cause all your conversions. Doesn’t happen.

So how does AI help with this? Through techniques that aggregate weak learners together and make them function as a stronger learner, we can get a sense of what combinations of dimensions and metrics matter most.

Hear Penn’s full answer to the question of how AI will enhance marketers’ ability to convert sales and retarget customers in the above video.

See Christopher Penn present the keynote session Using AI & Deep Learning to Generate Marketing Results at the FUSE Digital marketing Summit.

 

Do Buzzwords Get in the Way of Progress?

Have you read a column in the past week, month or year that’s void of buzzwords? Probably not. In the age of 5,000-plus choices of what partners, technologies or agencies to choose from, I find it uncanny how the marketplace is fraught with complex ways to explain simple things.

Have you read a column in the past week, month or year that’s void of buzzwords? Probably not. In the age of 5,000-plus choices of what partners, technologies or agencies to choose from, I find it uncanny how the marketplace is fraught with complex ways to explain simple things. Blame it on analysts who define industries? Blame it on a competitive marketplace and people trying to stand out with that killer phrase that describes what they do? Blame it on retailers striving to explain and justify what they do to their corporate leaders? Or startups striving to associate new ideas to mainstream challenges? Or blame it on consultants for making the simple complex and charging for it.

What it doesn’t help are retailers. In a perfect world, retailers live their brand. They look for simple ways to communicate with a broad spectrum of customers, and need creative yet practical approaches to words. You’re a merchandiser, an e-commerce company, and a lifestyle brand, and it can be a cultural challenge to balance buzzword frenzy with simple words the market needs to hear about your company. My main problem with buzzwords — and I’ve been as guilty as anyone in the use of them, just read a few of my columns — is using terms in loose context can minimize the impact of the term and make it actually more confusing. Therefore, in the spirit of no buzzwords, this column is just that: real talk for real retailers.

Lets start with a few buzzwords:

  • Disruptive technology: This begs the question of how disruptive your disruptive technology has to be for you to claim that it’s truly disruptive vs. just moderately irritating.
  • Ecosystem: This buzzword got big in mid-2014, 2015 as Luma Partners really promoted its Lumascape. Next thing you know every vendor is using it and every internal IT team began following suit to describe their “data lake strategies” and “technology road map.” I’m not sure I’ll ever get used to referring to my business interdependencies using the same terminology we use to talk about global warming and our attempt to save the planet.
  • Millennials: Are millennials really a buzzword? They might be. They’ve become more than just another generational grouping. As more millennials enter the workforce, replacing the retiring baby boomers, we will continue to spend a lot of time talking about the impact they’re having on the intersection between business, technology and our interpersonal lives. Maybe more importantly, we will continue to try to figure out why they break up with each other via text.
  • Thought leadership: This buzzword was prevalent for many years, and I still don’t really know what it means — or maybe I thought I did and really didn’t. I was awarded Thought Leader of the Year in 2016, and had trouble describing the award outside of … unfortunately, it seems to be entrenched and positioned to bother us for another year. I’ve been trying desperately to think of a new term that could supplant it, but question if I’m enough of a thought leader to make that happen.
  • Storytellling: I have to confess that I’ve coached and advised leaders to use stories to convey important things about their businesses because a good story resonates better than death by Powerpoint presentation. Now we’ve got storytelling classes, storytelling departments, and even storytelling gurus. Once gurus come into the picture, we’ve officially hit buzz status
  • Artificial intelligence/machine learning: These are likely the most overused, misunderstood and confusing buzzwords. How many times have you heard, “We have AI.” While this area of discipline and technology advances will reshape much of what we know today, any buzzword that conjures up impending doom of the human race isn’t helping in a dynamic business world.
  • Big data: I have trouble with anything that starts with “big” as a modifier of an industry trend. What’s big, and is there bigger? Much like the term disruptive, big data is an overused phrase that doesn’t serve many outside of its sellers. Google, Facebook, Amazon.com, Microsoft, Apple have big data. If you really want to understand big data in our society, there’s a great book: “The Human Face of Big Data.” Warning, this book is big, literally. In the end, the term does little to help you contextualize marketing problems or your own internal data challenges.

We’re in a world of endless information. Buzzwords in my opinion distort real talk and make complex concepts harder for the masses to address in situational marketing. Have fun with it by infusing a NO Buzzword culture or, better yet, force the offender to fully explain the term in the context of your business. And remember the goal of words is not to show how smart you are versus; they are a way to level set on complex ideas.

Make the complex simple!