![\"Dimensional](\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2022\/11\/Dimensional-Modeling-in-the-Advanced-Analytics-Age-1.png\")
<\/p>\nOver the past 5 years, the world of dimensional modeling and Business Intelligence has experienced major and disruptive technological change. Big data, machine learning, data science, deep learning \u2013 these terms and technologies are not all new, but they are now at the forefront of discussion for data practitioners.<\/p>\n
We\u2019ve seen the emergence of various Business Intelligence tools claiming to have either automated or replaced the need for data modeling. There has also been an explosion, both in terms of general interest and implementation, of new data storage technologies, such as data lakes. In today\u2019s climate, data modeling can be portrayed negatively, as a relic of the past.<\/p>\n
<\/p>\n
For Business Intelligence projects, data modeling usually means dimensional modeling \u2013 the approach was developed by Ralph Kimball<\/a>; this will be the heart of our discussion. Rather than rehash the merits of this approach over any other in the database structure context, let\u2019s examine its relevancy after recent upheavals in technology. To understand current best practices, we will start with a review of dimensional modeling\u2019s evolution.<\/p>\n Dimensional modeling can apply to any data practitioner \u2013 from a financial analyst who needs to create an executive dashboard in Power Pivot, to a data architect in a multinational company, or even anyone who spends more than 20% of their time manipulating data. If your job involves creating reports, dashboards, or basic forecasts for the year, month, and weekday, then dimensional modeling is relevant to you.<\/p>\n Let\u2019s go back in time and review why dimensional modeling was created in the first place, to discover if these arguments still hold true today. The initial goals were:<\/p>\n Fast forward 20 years and the original tenets underlying a dimensional model\u2019s use still hold true today.<\/p>\n Hardware and software have improved dramatically: a multi-month project, requiring a top-of-the-line server 20 years ago, can now be prototyped, with better performance, on a decent laptop with Power BI<\/a> in less than a week. The emergence of cloud computing also allows greater access to massively parallel databases for a fraction of the price. And today\u2019s big data technologies allow BI practitioners to manipulate a quantity of data that was unfathomable when dimensional modeling was created. While performance problems have improved, end users now expect less data latency. Most users won\u2019t accept having their finance data refreshed just once a month; this was the norm 15 years ago. Daily refresh, or even multiple daily refresh, is now typical, but is only achievable if the whole reporting infrastructure is optimized, proving dimensional modeling still plays a role in performance.<\/p>\n The integration capability of the bus matrix was probably one of the most important features of dimensional modeling. However, the theory often failed to deliver in practice. Rotating stakeholders with changing needs often muddied an enterprise\u2019s efforts to make use of its data. Additionally, the wait to add a new business process to the data warehouse often seemed too long \u2013 businesses often opted for quicker departmental solutions.<\/p>\n IT departments talked about data integration, while business units wanted to break departmental silos; though everyone agreed that solving these organizational issues was one of the most important data warehouse roles, success here was a challenge for most BI teams. Effective data modeling requires bringing the IT, marketing, and sales departments to the same table \u2013 to decide on common definitions and the skills required to create the databases, tables, and ETL processes \u2013 a capability rarely found together in BI teams of the past. Team members were typically either too technical or too functional.<\/p>\n The creation of hybrid teams, combining data engineers, business experts, and, with an increasing frequency, data scientists, is part of today\u2019s answer to the integration challenge. These hybrid teams will take various names in an organization: BI Center of Excellence, BI Competency Center. These cross departmental teams are often derived from the more generic Community of Practice<\/a>.<\/p>\n Master Data Management (MDM) projects, which use a systematic data integration approach, tackle the other challenges; MDM emphasizes governance and business processes. Integration is still a valid argument in favor of dimensional modeling, but perhaps more so it is an argument for MDM.<\/p>\n The main concepts comprising dimensional modeling are facts and dimension. Dimensions are (or should be) designed independently from their source system. The surrogate key, which identifies each member of a dimension, is independent from the source system. As a result, it can and should be re-used for different business areas. By designing each dimension (and as a result, the facts also) independently, the data warehouse is modular by design. This is still true. The advent of agile project management is only the confirmation that developing a data warehouse by manageable chunks is key to the success of a solution. The extensibility of dimensional modeling is still a relevant, key feature in favor of its use.<\/p>\n The recent dramatic evolution of technology has given rise to an exponentially complex data landscape. It includes semi-structured data, text analytics, sensor analytics, and web-related statistics. But if all of this information can\u2019t benefit its end users, it\u2019s useless to enterprise. And dimensional modeling still offers the best route to making sense of mounds of data.<\/p>\n Data is typically consumed through reports or dashboards built from tables or graphs. A well-built dimensional model can field nearly any related end-user query, in the form of an easily digestible flat table or graph.<\/p>\n Even seasoned report designers benefit from well-designed data models. Calculations are easier to develop, report creation is faster, and reports are more consistent from one developer to another. Dimensional models aren\u2019t just key to dashboards, reports, and simple data analysis \u2013 they also benefit data scientists.<\/p>\nLaying an Intuitive & Effective Groundwork<\/h2>\n
\n
\n
\n
\n
Stable Groundwork Still Key<\/h2>\n
Performance<\/h3>\n
Integration<\/h3>\n
Extensibility<\/h3>\n
Ease of Understanding<\/h3>\n