Avoid Concession of ETL{2}

by Jasmine C
The author of my articles discusses how ETL (extract, transform, load) teams try to compromise when implemented date warehouses and as the effect of this compromise is long-term problems.  If ETL teams did not cut corners then the ETL implementation would improve immensely. According to the article, the first compromise is “neglecting slowly changing dimension requirements” (Becker). SCD’s  provide important history context for a data warehouse and when ETL’s compromise on this step and push it to a later date, they loss that important history because they do not want to go back and  and rebuild the data warehouse.  The second compromise discussed in this article is “failing to embrace a metadata strategy” (Becker). ETL teams usually postpone capturing metadata until a future date because they do not own the metadata strategy. This is harmful because some of the most important metadata is captured in the beginning stages of implementation. “The ETL team only captures the information required for their development purposes, leaving valuable descriptive information on the cutting room floor” (Becker). These teams only use the information that they think is useful, however, in the long run, all the information is useful so at the beginning, they should just capture all the information because more likely than not, they are going to have to go back and recapture the required information anyway. The third compromise discussed is “not delivering a meaningful scope”  (Becker). This basically means that the ETL teams make compromises without telling anyone. In their designing phase, it is understandable for them to come up with compromise because these compromises are discussed openly and communicated with other project teams and sponsors.  When ETL teams take compromises without letting anyone know, to say the least,  they risk mishandling data quality errors fully testing processes, etc. This can then lead to inconsistent reporting, wrong data, and a lack of confidence in your business. These three compromises can lead to failure.

We were recently discussing data warehouses in class.  I believe this article is important to and relates to class because when implementing a data warehouse, you want to make sure that it is, to say the least, flawless.  It is okay to make compromises, every business does it.  These compromises are discussed openly so that everyone is on the same page. When creating a data warehouse, you do not want to make any mistakes in the beginning that in effect can mess up all the work you’ve done thus far. Communication and not taking any additional compromises is critical to a data warehouse.  I could relate this to our project because when we had to create SQL for the first time, we had to make sure that our ErWin model was correct. If it wasn’t and you later tried to add information to your database, you would get an error.  This is how I see taking compromises, don’t take them unless everyone is on the same page.

Becker, B. (2010, March 1). Kimball University: Three ETL Compromises to Avoid. Retrieved March 11, 2012, from InformationWeek: http://www.informationweek.com/news/software/info_management/223101036?pgno=1