Performance Tests in Data Warehousing ETLM Process for Detection of Changes in Data Origin

In a data warehouse (DW) environment, when the operational environment does not posses or does not want to inform the data about the changes that occurred, controls have to be implemented to enable detection of these changes and to reflect them in the DW environment. The main scenarios are: i) the impossibility to instrument the DBMS (triggers, transaction log, stored procedures, replication, materialized views, old and new versions of data, etc) due to security policies, data property or performance issues; ii) the lack of instrumentation resources on the DBMS; iii) the use of legacy technologies such file systems or semi-structured data; iv) application proprietary databases and ERP systems. In another article [1], we presented the development and implementation of a technique that was derived for the comparison of database snapshots, where we use signatures to mark and detect changes. The technique is simple and can be applied to all four scenarios above. To prove the efficiency of ...

• Type: Conference
• Year: 2003
• Where: DAWAK
• Authors: Rosana L. de B. A. Rocha, Leonardo Figueiredo Cardoso, Jano Moreira de Souza
• Keywords: Application Proprietary Databases, Data Mining, Data Warehouse, Data Warehouse Project, DAWAK 2003