Numerous studies show that companies that make their decisions based on figures and analysis are more successful than their competitors who give priority to personal experience and visceral decisions. Those who understand their customers’ behavior and motivation more accurately on the basis of systematic assessments, respond more specifically to their needs and at the same time react even faster than the competition, achieve better business results.
Without technical aids, the recognition of complex relationships in large amounts of data is not possible, especially not in real time. It is therefore not surprising that many marketing managers see “large data” analysis as one of the central challenges of the coming years.
Marketing has changed significantly over the past two decades. Where before it was characterized by the use of media and mass mailings to convince large groups of buyers of the benefits of a product, today the focus is increasingly on the individual customer.
Understanding them, their behavior and their needs – the much quoted 360° view of the customer – is the basis for deciding which product and which service to offer them through which of the increasingly available channels and under which conditions.
The more precisely customers are segmented and their behavior predicted, the higher the contact and response rates and ultimately the higher the closing rates and sales. But satisfaction and loyalty also increase due to less “dissonance” caused by a “wrong” customer approach. Through the focused and more successful use of budgets and resources, marketing becomes the primary driver of increased profitability.
However, what is so easy to read actually places high demands on the performance of IT systems and the quality of data analysis. Despite technological and organizational advances, the challenges have not diminished. The Internet and now ubiquitous mobile devices are massively transforming both the consumer market and the systems and processes of all businesses.
Social media are giving new impetus to the shift in power from the producer to the consumer, which has been observed for some time, and are even reversing the relationship forever. Customers are placing completely new demands on a company’s products and services. But they also expect intelligent and timely communication that is tailored to their own needs.
The acceleration of processes and entire markets is constantly reaching new dimensions. So is the amount and variety of data available for analysis. Analyzing “big data” requires a paradigm shift for many companies, especially if this has to be done in real time. But Massive Data also has opportunities.
There has never been so much valuable information about customers and markets. Forecasts are increasingly accurate and companies can identify and use much finer but significant correlations and target customer segments more specifically and quickly. Products and services can be tailored to the exact needs of customers (design to value) and thus significantly increase their satisfaction.
Customer Analytics – no chance without smart software
The amount, variety and complexity of data available today have made it virtually impossible for human analysts to identify the relationships they contain. It is extremely unlikely that new knowledge can be gained by examining the data, even if the patterns are comparatively simple.
At best, assumptions and assumptions can be specifically tested. But even these simple hypothesis-based confirmation analyses require tools that manage, aggregate significantly (mean values, sums) and visualize the information distributed in millions of data records. Many descriptive statistics and OLAP tools can no longer cope with the growth in data.
This also increasingly applies to relational database management systems, which over the decades have become the IT backbone of all large companies, but whose limits seem to have been reached for many purposes, for example in campaign management. Many companies are now adopting other approaches if, for example, they want to track the progress and success of their marketing campaigns despite the large amount of data.
If you have to look for previously unknown patterns in the mountains of data (exploratory analyses), you get lost without technical aids. Even relatively simple correlations can only be found randomly by simply navigating through the data. If the patterns become more complex, they are virtually invisible to even the smartest human analysts with the best software support for aggregating and visualizing the data.
This is where so-called data mining comes into play. Behind this group of analytical procedures is a multitude of algorithms and methods that, largely autonomously and automatically, detect significant correlations in large amounts of data, from which forecasts for future events can be reliably derived (predictive analysis). Modern methods are also capable of detecting effects that are the result of the interaction of dozens of influence factors. And they find these patterns in populations with tens of thousands of variables to millions and millions of data sets.
The fact that predictive models obtained in this way have more than a theoretical value is known for many years in marketing, where there are many applications for data mining. The models commonly used in database marketing often show a surprising ability to predict customer or market behavior or the success of product or service offerings with a high degree of probability.
Big Data Analytics – more than just an advertisement
For many years, marketers have relied on predictive analysis to identify complex relationships in large amounts of data and to predict customer behavior and market trends. But what used to be a large data set and a great challenge in previous decades, namely analyzing data on millions of customers and tens of millions of transactions, has now become the norm even for some medium-sized companies thanks to improved technology and established processing procedures.
But only as long as one does not pretend to analyze data in real time, as long as one limits oneself to considering aggregated information rather than individual transactions, and as long as many other sources of data and evaluations are not dispensed with. Performance-adjusted and elaborate database systems, but also time-consuming data processing and preparation processes make this possible.
However, the increasing demands of highly competitive markets and the growing amount and variety of data are forcing a change in the technological paradigm. When people talk about “Big Data” today, they mean data on tens of millions of customers and billions of transactions. But above all, we also refer to types of data that previously did not exist in this form and diversity or that for a long time were classified as virtually impossible to analyze.
These are mainly free text (such as the content of a website), but also images, video, audio data and – in the age of mobile devices and RFID technology – increasingly information about the whereabouts of people and objects. In addition, there is data that is fed back to the manufacturers by the increasing number of sensors contained in the products, thus enabling precise knowledge of the use, possible errors and current status of products such as machines, vehicles or software installations.
Macro Data has become a buzzword that manufacturers of business intelligence software and new types of hardware devices used extensively to drive sales of their respective products. In fact, the growing number of data records is just one of the key features that distinguish today’s business data from that of decades past. However, those who only consider this aspect can make poor investment decisions and try to offset the growth in data with even larger hardware or additional adjustment options, for example, when in fact qualitatively different approaches would be required.
In fact, with Big Data Processing at least the following aspects must be considered in individual cases:
- The diversity and otherness of current and future data sources compared to those of the past
- The present and future need for fast and appropriate actions and reactions in the contact with the client (and therefore the need for programs to make automated decisions based on intelligent algorithms)
- The tendency not only to underpin basic strategies through data mining, but also to optimize processes at increasingly detailed levels, which also means that analyses increasingly penetrate the details of data, ultimately even individual transactions, and that aggregations and pre-calculations are meaningless.
The Internet and mobile technologies and the new companies and business models that use them not only brought many new sources of data. They also gave rise to a multitude of new requirements and ideas for analysis. And in a world where everything should be available on the smartphone at the touch of a button and in real time, even analysis of the largest data holdings should no longer take days and weeks for data processing and statistical evaluation.
Therefore, Big Data Analytics is almost inevitably integrated into fully automated processes where human intervention is only required in exceptional cases.
Big Data – large amounts of data, new data sources
While classic database marketing essentially works with data structured within relational databases or flat files, around the turn of the millennium CRM systems initially brought new data sources with them: largely technical call center or web store protocols, which for volume and performance reasons are not included in transactional databases, but remain outsourced to files.
From the beginning, this information was of great interest for marketing and customer data analysis, since, together with the master and transactional data, it was very useful for understanding customer behavior and monitoring the success of marketing campaigns.
With the advent of Web 2.0, in addition to these mostly semi-structured text files, completely free text information from blogs, forums and social networks came into play more and more. The cloud: this data is no longer necessarily collected by company-owned systems, but is found on the Internet, where it is still available for analysis without restriction – as long as you know how and have an idea of what.
Ultimately, however, the Internet also contains non-textual information, especially audio, image and video data. The analysis of this type of data often requires completely different algorithms than those used in data mining until now. Today, many of the large data analyses still concentrate on relational and text data. But academic and industrial research is literally producing new methods for the analysis of non-text data on a daily basis. Non-text data analysis will gain in importance when a critical mass of intelligent ideas and profitable business models have grown up based on them.
Before that, however, it is likely to be the location information that is most useful in practice in Big Data Analytics. Providing people with the right services and information for their location – so-called location-based services – is no longer unusual, although the potential of these new types of service offerings is certainly far from exhausted. However, a wide variety of profiles can be created from people’s location information, e.g. movement profiles in the supermarket or within specific geographies.
There is also great potential in the analysis of product usage data, which can be transmitted to the manufacturer via software products or various types of machines and vehicles. In addition to the analysis of product defects, which is in the foreground here, this data can also be used to create usage profiles.
In the United States, for example, consideration is being given to making vehicle usage data available to insurers so that they can better assess risks. Of course, customized data collections and evaluations in particular must be considered very critically from a data protection perspective.
In classical database marketing – the basis of any campaign aimed at cross-selling or up-selling, churn prevention, new customer acquisition or customer recovery – models are essentially built on customer behavioral data and customer attributes. In the broadest sense, customer attributes are socio-demographic data (address information, age, status, etc.), supplemented by segment assignments or other classification characteristics that have been obtained from previous analyses, for example.
For a long time, customer behavior focused on specific sales data, usually grouped by product group and analyzed at different time intervals. Returns, discounts, and other order-related information often complemented the behavioral profile.
With the introduction of CRM systems and the advent of the online world, data on order “history” became increasingly available for analysis. The type and course of customer contacts in call centers and web shops became interesting, and the information about them can be considered the first major commercially relevant data.
However, few companies used these Call Detail Records (CDR) and clickstream data in a truly systematic way from the start, partly because they had to do more basic tasks elsewhere, but also because they lacked the tools to do so.
For many significant issues, this data must also be integrated with transaction and master data, for which no proprietary solutions exist and in many cases individual solutions must be found.
Predictive analytics in marketing after the millennium
The last decade has brought two important new trends, which have created new opportunities and, therefore, new demands for analysis: On the one hand, with the emergence of Web 2.0 with social networks, forums and blogs, the scope and importance of free text information increased massively.
The sales and marketing departments of many companies have quickly recognized that these new communication channels will change the market in the long term and that for many, especially young customer groups, they are the main way to exchange information with each other. Many of these customer groups can only be successfully served by this channel, as they no longer watch TV and barely read emails.
Modern consumer marketing is unthinkable without Facebook, Twitter and Co. Even in B2B business, forums and networks have become an essential part of public relations and lead generation on the one hand and support and service offerings on the other. However, the various varieties of Web 2.0 are not only relevant as a means of communication.
They also, and above all, offer unimaginable new possibilities for gaining knowledge about customers, who are often willing to disclose this information in exchange for relevant services. With all the critical aspects of data protection – some of which have not yet been conclusively clarified – there are certainly a great number of opportunities here for the benefit of companies and customers.
A frequently mentioned example is sentiment analysis, which automatically identifies moods and attitudes towards certain issues, products, companies, etc. Rumors and criticism circulating on the Internet can very easily cause permanent damage to a company’s image through silent messages and self-reinforcement effects.
If such risks are quickly recognized, the damage can be limited by corrections and a specific information policy. In some cases, a humorous and imaginative reaction can even lead to an increase in image. However, Web 2.0 sentiment towards one’s own products and those of competitors is also an early warning system that can announce possible declines in sales and the migration of larger customer groups.
Companies selling to consumers in particular can obtain indications in a relatively simple way about the direction in which future marketing campaigns should be directed, but also about the characteristics of products that will be in particularly high demand in the near future.
Big Data – also a question of technology
Most of the analysis tools used so far are not sufficiently equipped for the analysis of Web 2.0 data. They arose in the world of structured data and business processes, in which fields and their characteristics were conceived and whose specifications could be trusted, at least over long distances.
However, they do not offer the technical access to unstructured data and, above all, they do not offer the analytical access to the semantic finesse that resonates in blog and forum posts and that can even represent the actual statement. Synonyms, word plays, word creations, creations of new acronyms, humor – these are all aspects of language and human thought that analytical tools must now learn to address in order to extract useful information from text data for specific questions.
The technical methods of text mining and search engine technology have found their way into business intelligence solutions and can answer the questions mentioned above. It is expected that in the coming years more and more modules for image processing, speech and language recognition and many other areas will be incorporated into the analysis infrastructures.