{"id":10328,"date":"2020-04-30T00:00:00","date_gmt":"2020-04-30T05:00:00","guid":{"rendered":"https:\/\/threecloud.wpengine.com\/post\/slowly-changing-dimensions-in-data-modeling\/"},"modified":"2022-11-30T09:24:44","modified_gmt":"2022-11-30T15:24:44","slug":"slowly-changing-dimensions-in-data-modeling","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/slowly-changing-dimensions-in-data-modeling\/","title":{"rendered":"How to Handle Slowly Changing Dimensions"},"content":{"rendered":"<p>Data modeling is surely not a new concept, but it is a key one, especially in the age of big data. \u00a0<strong>Data modeling emphasizes what data is needed and how it should be organized. Let\u2019s talk about everyone\u2019s favorite topic, Slowly Changing Dimensions.<\/strong><\/p>\n<p>&nbsp;<\/p>\n<div class=\"hs-responsive-embed-wrapper hs-responsive-embed\" style=\"width: 100%; height: auto; position: relative; overflow: hidden; padding: 0; max-width: 560px; max-height: 315px; min-width: 256px; margin: 0px auto; display: block;\">\n<div class=\"hs-responsive-embed-inner-wrapper\" style=\"position: relative; overflow: hidden; max-width: 100%; padding-bottom: 56.25%; margin: 0;\"><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/qrBfD1YjO1I\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/div>\n<\/div>\n<p><strong>A dataset typically contains your fact tables and dimension tables.<\/strong> Dimensions define what you want to analyze or to slice your facts by, such as calendar or geography with their hierarchies (customers, stores, employees, sales channels, etc.). <strong>Fact tables change very rapidly; dimensions stay pretty consistent.<\/strong><\/p>\n<p><strong>Sometimes the data in those tables can change as well. How we handle those changes is what I want to talk about.<\/strong> There are three types of changes but I\u2019m going to focus on the two changes that are most common.<\/p>\n<ul>\n<li><span style=\"text-decoration: underline;\"><strong>Type 1 Slowly Changing Dimensions<\/strong><\/span> \u2013 This type occurs when we want to overwrite the data and it\u2019s not necessary to preserve the history. In other words, when changes in data types are detected, a type 1 change always reflects the current values.<\/li>\n<li>This can happen when there\u2019s a data error that needs to be corrected, such as an incorrect entry of an employee birth date. We can go into the employee dimension table, update the birth date there and we are done. We don\u2019t care if the historical records have changed or any reporting has been affected; our goal is just to correct the error.<\/li>\n<li><span style=\"text-decoration: underline;\"><strong>Type 2 Slowly Changing Dimensions<\/strong><\/span> \u2013 This type is a bit more complex as we need to preserve the history. For example, if a person changes their address, location, or name, how do we want to handle that from the data perspective?<\/li>\n<li>In this case we need to make sure there is versioning of the dimension numbers. Let\u2019s say our customer, Sally, has moved to a different state and changed her address. We need to have two versions of Sally depending on when the change occurred.<\/li>\n<li>As you can see below Natural Key, such as social security number, won\u2019t help us here.<\/li>\n<li><img decoding=\"async\" style=\"width: 443px;\" src=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2021\/09\/2020-04-24_20-00-50.jpg\" alt=\"2020-04-24_20-00-50\" width=\"443\" \/><\/li>\n<li>We need a surrogate key that identifies each version of Sally. So, if we need to run a report of shipments by state, we will appropriately show shipments to the same customer going to two different states.<\/li>\n<li><img decoding=\"async\" style=\"width: 527px;\" src=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2021\/09\/2020-04-24_20-01-20.jpg\" alt=\"2020-04-24_20-01-20\" width=\"527\" \/><\/li>\n<li>If the source system doesn\u2019t store versions and you have a data warehouse, the data warehouse load process will detect those changes. If there\u2019s no data warehouse this is something to be aware of as you\u2019re building your reporting solution.<\/li>\n<li>It would also be a good idea to add the range validity of the version of Sally, such as start and end data columns as well as a flag column which states which is the current version of Sally.<\/li>\n<li><img decoding=\"async\" style=\"width: 824px;\" src=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2021\/09\/2020-04-24_22-37-10.jpg\" alt=\"2020-04-24_22-37-10\" width=\"824\" \/><\/li>\n<\/ul>\n<p>Hopefully, you found this quick post on Slowly Changing Dimensions and how to handle data changes helpful.<\/p>\n<p><strong>Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or\u00a0 <a href=\"mailto:sales@3cloudsolutions.com\">sales@3cloudsolutions.com<\/a>.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data modeling is surely not a new concept, but it is a key one, especially&mldr;<\/p>\n","protected":false},"author":43,"featured_media":10829,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[260],"tags":[],"class_list":["post-10328","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-ai","topics-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10328","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/43"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=10328"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10328\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/10829"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=10328"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=10328"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=10328"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}