A data warehouse is very useful and valuable as a data source for report generation, dashboards and advanced analysis in an organization that has relatively many systems or applications. The data warehouse will store consolidated data from various applications so that reporting and analysis can be much faster and easier.
Without a data warehouse, report makers have to manually retrieve data from each data source and this is often frustrating. The frequency of filling or updating the data warehouse from data sources depends on business needs but is generally daily. Data warehouse is synonymous with structured data; but currently unstructured data, such as documents, web pages, social media are also objects of the data warehouse. Data warehouse development generally includes designing data models according to business needs and making ETL (Extract, Transform and Load) processes. In addition to ETL, there is another approach, namely ELT (Extract Load Transform) where the Transform process is carried out as close as possible to the data storage area to reduce data transfer through the network. ETL and ELT carry out the process of retrieving data from the source, transforming data according to the form of the destination data model, calculating according to business needs, to writing or loading the transformed data to the destination.
The technologies used in building the data warehouse include ERwin, IBM Data Stage, Talend, Pentaho Data Integrator, Teradata, Oracle, SAS, etc. Knowledge and experience of the business is also very influential on the results of the data model design. The data warehouses that have been created include sales data marts, loan data marts, credit card data marts, etc.