Optimization of Data Warehousing System: Simplification in Reporting and Analysis

Abstract

A data warehouse maintains its functions in three layers: staging, integration, and access. Staging is used to store raw data for use by developers (analysis and support). The integration layer is used to integrate data and to have a level of abstraction from users. The access layer is for getting data out for users.

This definition of the data warehouse focuses on data storage. The main source of the data is cleaned, transformed, catalogued and made available for use by managers and other business professionals for data mining, online analytical processing, market research and decision support. However, the means to retrieve and analyze data, to extract, transform and load data, and to manage the data dictionary are also considered essential components of a data warehousing system. Many references to data warehousing use this broader context. Thus, an expanded definition for data warehousing includes business intelligence tools, tools to extract, transform and load data into the repository, and tools to manage and retrieve metadata. Data Warehouse plays an important part in the process of knowledge engineering and decision-making for Enterprise, as a key component of the data warehouse architecture, the tool that support data extraction, transformation, loading is a critical success factor for any data warehouse projects.

Reference

• Simitsis, A.; Vassiliadis, P.; Sellis, T.; “State-space optimization of ETL workflows”, Knowledge and Data Engineering, IEEE Transactions on Volume:17 , Issue: 10 Digital Object Identifier: 10.1109/ TKDE.2005.169 Publication Year: 2005, Page(s): 1404 –1419.
• Zhuolun Zhang; Sufen Wang; “A Framework Model Study for Ontology-Driven ETL Processes”, 4th International Conference on, Wireless Communications, Networking and Mobile Computing, 2008. WiCOM '08. Digital Object Identifier: 10.1109/ WiCom.2008.2651Publication Year: 2008, Pages: 1- 4
• Tang Jun; Cui Kai; Feng Yu; Tong Gang; “The Research & Application of ETL Tool in Business Intelligence Project”, International Forum on Information Technology and Applications, 2009. IFITA '09. Volume: 2 Digital Object Identifier: 10.1109/ IFITA.2009.48 Publication Year: 2009, Page(s): 620 - 623
• Xiaoyun Liu; Jianhua Chen; Yingchun Zha; “Design of ETL Aggregation Policy in Local Tax System (LTS)”, International Conference on Management and Service Science, 2009. MASS '09. Digital Object Identifier: 10.1109/ICMSS.2009.5300877 Publication Year: 2009, Page(s): 1 - 4
• Xudong Song; Xiaolan Yan; Liguo Yang; “Design ETL Metamodel Based on UML Profile”, International Symposium on Knowledge Acquisition and Modeling, 2009. KAM '09. Second Volume: 3 Digital Object Identifier: 10.1109/KAM.2009.112 Publication Year: 2009, Page(s): 69 - 72
• Simitsis, A.; Wilkinson, K.; Dayal, U.; Castellanos, M.; “Optimizing ETL workflows for fault-tolerance”, International Conference on Data Engineering (ICDE), 2010 IEEE 26th Digital Object Identifier: 10.1109/ICDE.2010.5447816 Publication Year: 2010 , Page(s): 385 - 396
• Lunan Li; “A framework study of ETL processes optimization based on metadata repository”, 2nd International Conference on Computer Engineering and Technology (ICCET), 2010 Volume: 6 Digital Object Identifier: 10.1109/ ICCET.2010. 5486338 Publication Year: 2010 , Page(s): V6-125 - V6-129
• Morris, H.; Hui Liao; Sriram, P.; Srinivasan, S.; Lau, P.; Shan, J.; Wisnesky, R.; “Bringing Business Objects into Extract-Transform-Load (ETL) Technology”, IEEE International Conference on e-Business Engineering, 2008. ICEBE '08. Digital Object Identifier: 10.1109/ ICEBE.2008.72 Publication Year: 2008,Page(s):709-714.

Index Terms

Key words