Data warehouses are complex creations. The ETL and data cleansing processes that sanitize and reshape the data, the relational database in which the data warehouse resides, the auditing routines verifying that the data is correct, and the reporting and analytics tools that sit atop the entire structure all come together to make an intricate but immensely valuable business asset. In fact, most of the effort and time on the project schedule are focused on the technical components of a data warehouse solutions.
As with any development project, careful attention must be paid to using best practices in putting together the bits for the solution. However, with data warehouse projects, too often the focus becomes the design and behavior of the technology. Focusing just on the technical aspects of a data warehouse solution is effective for heads-down coding of specific pieces, but does not work for managing the project as a whole.
When building a data warehouse, there is one critical point that must always be kept front of mind:
A data warehouse is a business initiative, not a technical one.
I’ve seen data warehouse initiatives go off track when the focus shifted away from the business needs. Through every aspect of the project – initial brainstorming, technical design, testing, validation, delivery, and support – the needs of the business must drive every decision made. While the data warehouse will be built using memory, disks, code, tables, and ETL processes, the primary goal of the project must remain clear: The data warehouse exists to answer business questions about the data. Anything contradictory to that is a barrier and must be removed.
When architecting a data warehouse solution, build it using the best technical design possible. But in all design decisions, remember the ultimate goal and audience of the final product. Data warehousing is about the business, not the technology.