Ten years ago, the average storage cost per gigabyte of data was $1.24.
Now, it’s just under $0.03.
Data has never been easier or cheaper to collect and store... but that mean businesses are making better decisions because of it?
The term “big data�? is used to refer to many things – usually massive data sets of sales and customer characteristics. Such data can aim to predict customer sales patterns, clarify customer segmentation, link customer characteristics to purchasing patterns, and, for more sophisticated users, identify the relationship between marketing activity and revenue over time (Mela et al. 1998).
The idea of big data is certainly compelling - uncover hidden patterns, predict future consumer trends, and tailor your strategy accordingly. When it comes to historical trend analysis and predictive modeling, big data can be extremely valuable. For established companies with the resources to maintain and analyze big data, the resulting knowledge and increases in efficiency can be well worth the cost.
Data-driven everything
A large part of the big data trend is the move towards data-driven decisions, especially in marketing. In fact, 65% of senior executives say that their management decisions are increasingly based on analytics. Another study showed that 48% of C-level executives say that customer insights and targeting take priority for applying big data.
But it’s not always as easy as it sounds. For newer or smaller companies especially, the deluge of data can be overwhelming. More importantly, it is not necessarily the best source of information for all business decisions.
Not all data is equal
When you have thousands of data points for each individual customer, it’s easy to think that you have the data you need to answer any question and it’s just a matter of cutting it the right way. Today, there is an abundance of data capturing every consumer interaction in almost every company.
However, although analyzing clickstream data might help you make adjustments to a landing page, it might not have anything useful to say about how a subset of your customers will respond to a particular product.
In today’s data-heavy world, we face the risk of making the data fit the decision, when it should be the other way around. Truly effective data-driven decision-making starts by establishing what data you need to make an informed decision, instead of working backwards from what you already have.
In a study by The Economist, 76% of CEOs viewed big data favorably but only 48% considered it a useful tool. There are many circumstances where big data is the most appropriate tool, but it’s important to remember that it is not the only source of data available.
The Limitations of Big Data
In the midst of the big data hype, a lot of businesses are opting to collect every data point they can, but analyzing and learning from these colossal data sets can be a difficult exercise. More data does not automatically equal more information. Informed decisions require context, and that is where small data comes in.
Big data provides insights into a plethora of inputs and outputs, potentially connected with one another. It excels at providing guidance for spending and pricing decisions. For example, data and models can help us to know that we can drive more sales with price promotion in the last week of the month than in the first, or that certain media are more efficient than others for generating revenue.
What it isn’t so good at is revealing what impacts customer behaviors and choice. Understanding customer behavior is a mysterious “black box�? for brand owners, who try to gain glimpses of what’s inside by pulling together multiple data sets that may hint at the underlying drivers of customer choice. However, consumption data and even customer satisfaction or loyalty data only show a piece of the puzzle.
Worse, executing tactics on big data insights run the risk of alienating customers who, for example, tire of bulk e-mails that speak to the sender’s promotional interests rather than their own needs.
And finally, there is the pain of even getting the right data. Brand owners find themselves drowning in data but dying of thirst when it comes to insights that help them improve their outcomes. In fact, an eConsultancy survey found that 52% of marketers reported that data wasn’t always available to them, resulting in blind spots. Further, 39% complained that data was difficult to obtain, and 24% reported their data was stale. In other words, while they have more data at their fingertips than ever before, it’s not the right data to improve their decision-making. It turns out that knowing how customers make choices is essential for improving almost every business decision and big data tends to provide little guidance on the topic.
Introducing small data
To fill the gap, we need a new type of insight which falls into the camp of small data.
Small data is what we get from the study of how particular customers choose between competing offerings in certain contexts. What is most important to their choice, and how do each of the offerings in the market compare to one another? This small data connects the dots, opens up that “black box�? of customer purchasing behavior, and drives data-driven decisions that are aligned with customer needs.
Case Study
Consider Chris, a working mother who handles most of the consumption decisions in her household. A consumer packaged goods firm that makes aspirin can know a great deal about Chris or people like her via big data. There are demographic breakdowns of purchase frequency and brand-switching behavior, Google Analytics data to identify web search trends by demographics, and clickstream data to identify online advertising response.
But, for all of this data, there is no way to understand why Chris, mother of four and operating on extreme time and end-of-the-month budget constraints, chooses to buy the 100-count bottle of the national brand of aspirin for $5.96 and not the 500-count store brand for $3.64 – despite the fact that Chris believes, based on a Google search of research evidence and U.S. Food and Drug Administration (FDA) reports, that the store brand is just as effective.
While an economist would question Chris’ rationality, a deeper study of the factors driving her choice behavior would reveal a very simple set of explanations. She is buying the aspirin for her husband, just advised by his doctor after a check-up to take one aspirin a day for circulation.
When a family member’s health issue is on top of mind, stakes go up.
Further, the national brand was always in the medicine cabinet at her home when she was a little girl. Chris has a deep and abiding trust in the brand, as her mother was always a loyal purchaser. Finally, although her husband has never had a stomach problem from using aspirin, she notes that the national brand has a coating, which she believes will make it safer for everyday consumption.
Big data could not reveal the opportunity for coated aspirin. Neither can big data track with any precision the often rapidly changing perceptions of consumers as they choose among brands in categories where brand variety changes frequently. There are many significant actionable insights that emerge from the study of choice behavior, having implications for all 4 P’s: product/service, price, place, and promotion.
Cost vs. benefit of insights
Big data is expensive, and it can be time consuming to extract the metrics you are after or even know where to start in some cases. On the other hand, small data (such as customer choice data) is relatively inexpensive and does not require deep modeling expertise for analysis.
This is an important consideration given that nearly two thirds of surveyed executives stated that they are dissatisfied with their staff’s quantitative analytical skills, which might partly explain why 65% of CEOs agreed that their companies are only able to interpret a small proportion of accessible data.
Another key point is the comparative agility of small data. It can be collected and turned into useful insights within a short time frame, enabling organizations to adapt their strategy in the moment. Even the most detailed data analysis may not give you the same clarity as actually talking to your customers.
Big data is changing the way companies do business but in order to make the decisions that deliver the most value to consumers, small data needs to play a bigger role. Business don’t necessarily need more data, they instead need the right data. Data that will help them quickly make the decision they need to make.
In fact, in the end, the best strategy is to identify the best balance of integrating big and small data. That is a powerful combination.
