{"id":10308,"date":"2019-02-26T00:00:00","date_gmt":"2019-02-26T06:00:00","guid":{"rendered":"https:\/\/threecloud.wpengine.com\/post\/how-and-why-to-add-source-control-repository-to-azure-data-factory\/"},"modified":"2023-05-01T15:49:26","modified_gmt":"2023-05-01T20:49:26","slug":"how-and-why-to-add-source-control-repository-to-azure-data-factory","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/how-and-why-to-add-source-control-repository-to-azure-data-factory\/","title":{"rendered":"How and Why to Add a Source Code Repository to Azure Data Factory"},"content":{"rendered":"<p>For developers, it&#8217;s very beneficial to have a source code repository. <strong>A source code repository helps to keep all your changes, to manage tasks, branches, share the code with a team and simply put, to keep it in safe place.<\/strong><\/p>\n<p><strong>In this post, I\u2019ll tell you why you should connect your Azure Data Factories to a source code repository, and I\u2019ll demo how to do so:<\/strong><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>To do this, I\u2019ll start in my data factory inside my Azure portal.<\/li>\n<li>When I go into Author &amp; Monitor, I have the ability to either:\n<ul>\n<li>Set up a code repository within the landing page or main page here or,<\/li>\n<li>I can go directly into my data factory and I can add it in the left-hand corner and pull down to where it says, \u2018set up code repository\u2019.<\/li>\n<\/ul>\n<\/li>\n<li>One thing to note is the code repository itself has been supported for a while, but recently with the release of Data Flows, they\u2019re now supporting GitHub within the repositories as well (previously is was only Azure DevOps).<\/li>\n<li>So, I select my GitHub account and fill in the information. *If you\u2019re doing this for the first time, it\u2019s going to prompt you to log into your GitHub account when you do this. In this case, I\u2019ve already previously connected this so it\u2019s going to know about my repositories.<\/li>\n<li>Next, I select my repository name and I\u2019ll go to my playground branch and I\u2019m going to use my existing playground.<\/li>\n<li>One field will ask me \u2018Branch to import resources into:\u2019 so if I\u2019m importing resources, I can select an existing one or create a new one. For this demo I\u2019m going to pick my playground.<\/li>\n<li>Before I hit Save, notice on the left-hand side I\u2019ve got zero pipelines, one data set and zero data flows. But when I connect to my playground, it\u2019s going to bring everything in I\u2019ve previously connected to within any of my areas I\u2019ve saved up into or checked my code into anything in that playground branch. So, now you\u2019ll see I have 6 pipelines, 23 data sets and 4 data flows.<\/li>\n<li>One of the other nice pieces of being able to add source control is if I want to add a new data set. I just select my SQL Server I was previously connected to; I leave it on default for now and connect in.<\/li>\n<li>I then select one of the tables; I selected the Product Category Table.<\/li>\n<li>You\u2019ll see at the top you have the option to Save All or Publish. Save All is going to save any of the changes across the tabs, so you can tell when there\u2019s been a change, whether it be data set or pipeline or data flow by having a star next to the name.<\/li>\n<li>Now instead of needing to publish every time you\u2019re doing development, you can just save it and it will save here. So, rather than having to publish the entire pipeline and do any of the error checking and make sure the pipeline is in good standing, you can now just save part way through. This is a huge advantage over having to publish the entire pipeline which could cause some challenges which might not be efficient for development and such.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/Kb9JAgYRPWc\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><strong>The new features of GitHub being added in gives us another great opportunity if you didn\u2019t previously use Azure Dev Ops (formerly known as Visual Studio Team).<\/strong> I\u2019ll be doing more upcoming blogs around Data Factory in Azure Every Day that will be beneficial to you with some of the nuances as the product has greatly evolved since releasing Data Flows.<\/p>\n<p>Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or\u00a0 <a href=\"mailto:sales@3cloudsolutions.com\">sales@3cloudsolutions.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For developers, it&#8217;s very beneficial to have a source code repository. A source code repository&mldr;<\/p>\n","protected":false},"author":21,"featured_media":9366,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[381,292,260],"tags":[385],"class_list":["post-10308","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-app-experience","category-app-innovation","category-data-ai","tag-devops","topics-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10308","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=10308"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10308\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/9366"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=10308"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=10308"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=10308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}