Data Warehouse: Modernization or Reconfiguration? - 【2020】
data warehouse modernization

Data Warehouse: Modernization or Reconfiguration?

Do your company’s specialized departments have to wait a long time for BI evaluations? Do you want to carry out IO projects or develop new business models? Then it’s time to check whether your data warehouse still meets today’s requirements – and is ready for future ones.

The ability to evaluate the data and draw conclusions from it is crucial for business success today. The prerequisite for this is a solid but agile Data Warehouse (DWH). But in many companies, the central data platform has become outdated and can no longer meet today’s requirements.

When departments have to wait days for critical reports and work with outdated data because performance or stability leaves something to be desired, this is not sustainable. Companies are now faced with the question of whether it is better to modernize the existing Data Repository or build a completely new one. There are both technical and strategic aspects to consider. Firstly, you should check whether your data warehouse is still up-to-date and meets the requirements of the departments. If complaints about BI reports taking too long accumulate, this indicates poor performance. This may be related to outdated hardware, as well as database or data modeling problems.

Read also
Data Warehouse: Solutions for Small Businesses

Even if stability problems occur, this makes daily work difficult. After all, in order for specialist departments to be able to access the latest figures, the data warehouse must be continually fed with new information. If the reloading process ends, no current BI reports can be created. Low performance and instability are the most common reasons why a data warehouse must be modernized.

Less obvious, but still serious, are problems with data integrity and historicity. They lead to distorted results. Have you ever found that data in the data warehouse does not match the data in the source systems? If, for example, the total turnover shown in the Data Storage differs from the aggregated individual postings in the financial accounting system, it is possible that deductions or cash discount allowances have not been correctly allocated. Data integrity problems are often related to the fact that key figures and processes are not sufficiently documented in the data warehouse. Especially with complex structures that have developed over the years, it is no longer possible to understand how a key figure is actually produced.

Read also
What is Real Time Data Warehouse?

Problems with historicization occur when master data changes but no time reference is recorded in the data warehouse. This leads to historical data being assigned incorrectly. For example, if a customer moves from a neighborhood to New York, all his previous actions will suddenly be applied to the new New York address, even though they took place in his previous location. Errors in historicization are always related to data modeling. To prevent this from happening, those responsible must create the appropriate attributes for the master data in the data warehouse and also take the time reference into account.

Too much complexity as a criterion for elimination

A big problem of an old data warehouse is the complex structures that have grown over the years. They affect the performance, stability and integrity of data, as well as opportunities for future development. When the data warehouse was built, it met today’s needs. But over the years, new requirements have been continually added.

Read also
Difference Between Data Warehouse, Data Mining and Big Data

As a result, the data warehouse has been constantly expanded and adapted. However, like a house once built, it is not possible to tear down arbitrary walls in a data warehouse. Sometimes complicated solutions are required to integrate a new key figure into the already developed structure. This costs time, and departments may have to wait a long time until their new requirements are implemented.

With each change, the structure also becomes more complex and confusing. Sometimes data is saved twice because it cannot be reused. This makes it difficult to understand the data used for each calculation. In addition, if this data is not at the same level, the results will be different. If a data warehouse has become very complex and confusing, it may be better to rebuild it from scratch rather than modernize it.

In principle, a data warehouse should be structured so that it can be expanded and is properly documented. All the data you need must always be in the same place. If these requirements are met, future requirements of specialist departments can also be integrated without much effort. In addition, it is possible to add or exchange BI front-end tools to the data warehouse as required, so that departments can create reports independently with a click of the mouse. This saves time and relieves the IT department.

Read also
Different Layers in Data Warehouse Architecture

The sustainability of DWH depends on strategic issues

The fact that the data warehouse is sustainable is also related to strategic issues. Where does your company want to develop in the coming years? Anyone planning Io projects needs the ability to store and process huge amounts of data from various sources and in various formats, including both structured and unstructured data, such as text documents or videos. This may require connecting the data warehouse to a data lake and using tools that allow access to both data platforms.

Anyone planning to change their business model should also consider the data warehouse. If, for example, you no longer want to sell machines in the future but offer them for rent “as a service”, then you need to collect completely different data and map it into the data repository – for example, information about where a machine is currently located, how many hours it has been in operation at the customer site, what condition it is in and when it needs to be repaired.

Read also
Data mart vs Data Warehouse: Differences

The Data Warehouse should be a matter for the boss

Especially the question of future requirements shows that data storage is not just an IT issue, but is closely related to corporate strategy. Therefore, a member of the management must always be involved in the future of the data platform.

It is worth bringing the expertise of an experienced consultant into the company. In workshops and joint interviews, because it determines the current and future needs. A thorough analysis of the actual situation and the objective makes it clear whether it is worth modernising the existing data warehouse or whether it is better to rebuild it immediately.