BI – Business Intelligence is an aging term. Its approach (mostly backwards oriented) provided only limited competitive advantages and, like DWH, did not transform the business.
Instead, Data Analytics and BigData do.
Data warehouses were structured data collections from various sources. Today’s big data is a mix of structured and unstructured data, but that’s not the only thing.
You can easily explain the difference by comparing the “data types” and “team composition” of DWH and BigData.
Data types of the Data Warehousing era: customer profiles, contacts, marketing information, orders, quotes, deliveries, billing data, complaints, call center data.
Data types of the BigData era: purchase history, customer segmentation, customer value, purchasing behavior, recommendations, sentiment analysis, target marketing, customer satisfaction, customer experience management, customer lifecycle management
Positions in Data Warehouse: Database (IT) Developer, BI Specialist (IT), Business Analyst Sponsor: CIO or CFO
Jobs at BigData : data scientist (business, often biologist or physicist), graphic designer (visualization), developer (IT or business), customer representative (business champion). Sponsor: CEO or Head of Analysis The challenge in staffing is the ability to have skills in the team that can derive (structured) knowledge and business instructions from unstructured content.
Of course there are many other differences. At BigData, the focus is more on the visualization of data than on the creation of lists of figures.
Data Warehouse vs Business Intelligence?
Business Intelligence (BI) and Data Warehousing (DWH) are not projects that are defined, implemented or completed. It is a continuous process that must be deeply rooted in the corporate culture and be in harmony with other corporate processes. Those who follow this high principle will be successful in introducing BI/DWH.
Business Intelligence is representative of the different forms of assessment tools and methods, as well as Business Analytics, Advanced Analytics, Data Mining or even self-service BI. The term covers all methods of analysis and information in the company, with the main purpose of answering business questions, from standard reports in Controlling to pattern recognition of weblogs in the area of Customer Journey.
The data warehouse as an information base for evaluations comprises data storage, data preparation and data quality management, extended with an additional information base for the collection of structured and unstructured data of various formats, the data lake. The Data Lake is the basis for exploratory analysis procedures. The structured results of the analysis procedures serve in turn as the source of a data repository.
One cause the other (BI – DWH)
The whole process can be compared to an iceberg. What is visible to business users with business intelligence and analysis tools is only a fraction of the overall picture and represents about 10 to 20 percent of the effort required for implementation. The larger remainder (DWH) includes source connections, harmonization, layered data processing, and implementation of topics such as data quality, compliance, and master data management. Therefore, the major implementation effort in this area, invisible to the user, is below the water surface.
The self-service BI approach seeks to break this principle to give the experienced business user more flexibility to connect and link any source. This flexibility, using self-service tools and analytical instruments, allows new knowledge to be gained and, if necessary, transferred to a data storage. However, the company’s information base as a “single source of truth” should be available in a quality assured data warehouse.
Processing automation with standardized ETL processes in all layers of a DWH enables the business user to access prepared and structured information that is regularly comparable, structurally harmonized, and professionally tested. This ranges from uniform key performance indicator (KPI) systems to rule-based data mining in DWH and Data Lake.
- what is DWH harmonization layer