{"id":10321,"date":"2019-09-04T00:00:00","date_gmt":"2019-09-04T05:00:00","guid":{"rendered":"https:\/\/threecloud.wpengine.com\/post\/how-to-connect-azure-data-factory-to-azure-devops\/"},"modified":"2023-05-01T15:49:50","modified_gmt":"2023-05-01T20:49:50","slug":"how-to-connect-azure-data-factory-to-azure-devops","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/how-to-connect-azure-data-factory-to-azure-devops\/","title":{"rendered":"How to Connect Azure Data Factory to Azure DevOps"},"content":{"rendered":"<p>Are you using Azure DevOps and want to know how to use it as a code repository? <strong>A benefit to using DevOps (or any code repository) is you can create a method to preserve the code from a working version while you\u2019re making modification. In this post I\u2019ll show you how to connect an existing Azure Data Factory project to an Azure DevOps code repository.<\/strong><\/p>\n<\/p>\n<p>Azure Data Factory (ADF) uses JSON to capture the code in your Data Factory project and <strong>by connecting ADF to a code repository each of your changes will be tracked when you save them. Also, whenever you publish, DevOps will automatically establish a new version of the Data Factory, enabling you to rollback if needed.<\/strong><\/p>\n<p>Now on to my demo:<\/p>\n<div class=\"hs-responsive-embed-wrapper\" style=\"width: 100%; height: auto; position: relative; overflow: hidden; max-width: 560px; max-height: 315px; min-width: 320px; margin: 0px auto; display: block;\">\n<div style=\"position: relative; overflow: hidden; max-width: 100%; padding-bottom: 56.25%; margin: 0;\"><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/fCiTthat1zk\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/div>\n<\/div>\n<ul>\n<li>I\u2019ve created a simple Data Factory that counts and copies an author cable from an Azure SQL Database to an Azure Storage Blob.<\/li>\n<li>Azure DevOps supports two versions of a code repository: Azure DevOps and GitHub. In this demo, I\u2019ll work with Azure DevOps.<\/li>\n<li>First step is to log into Azure DevOps (dev.azure.com) then click on New Project, fill in the default fields and click Create. You could go also into Advanced and change your version control and work item process (I\u2019ll work in Agile).<\/li>\n<li>While that\u2019s creating, I\u2019ll go back into my Data Factory. In the upper left corner, you\u2019ll see Data Factory and when you pull that drop-down, click on Set Up Code Repository.<\/li>\n<li>This will open Repository Settings where we can set up the connection to the code repository we just created. We need to:\n<ul>\n<li>Select Repository Type: Azure DevOps Git<\/li>\n<li>Select Azure DevOps account that it\u2019s associated with (my account in this case)<\/li>\n<li>Choose Project Name (the one we just created)<\/li>\n<li>Git Repository Name: We can create a new one or use the existing repository when we created it.<\/li>\n<li>Collaboration Branch: I suggest you stick with Master. This is where all your branching will merge back, as well as where a copy of all the changes you\u2019ve made will be published to the Azure Data Factory that runs, via trigger or event.<\/li>\n<li>Then click Save.<\/li>\n<\/ul>\n<\/li>\n<li>While this is saving, you\u2019ll see on your ADF page that Save as Template is grayed out but underneath that you\u2019ll see 2 new Save buttons pop up. Those will allow you save the changes you made which is different than what you used to do which was to publish them to the Data Factory.<\/li>\n<li>You\u2019ll now have Saved, Save All and Publish Buttons. Additionally, you\u2019ll be asked what branch you want. You could create a new one, but I choose the existing (master) branch.<\/li>\n<li>At the top you\u2019ll see it noted that you\u2019re working out of the master branch and Azure DevOps GIT. If you go there and try to go back and select Azure Data Factory, you\u2019ll get a warning, publishing in Data Factory mode has been disabled, as we chose the DevOps GIT as our branch in this case.<\/li>\n<li>Next we want to create a new branch and it suggests I create that under my name. This can be good so when you\u2019re working with others, they\u2019ll see your branch name and know what you\u2019re working on. I\u2019ll start here but you\u2019ll see my make a change here in a bit.<\/li>\n<li>In my demo, I\u2019ll add in a wait command, so we can see how the change gets captured. In Get Number of Rows, I\u2019ll choose to make this wait happen when we get a failure, then I connect the failure to the wait and click Save.<\/li>\n<li>When I hit Publish, I get an error message that says \u2018publish is only allowed from collaboration (master) branch. Merge the changes to master.\u2019<\/li>\n<li>To merge my changes to master, I go up to where is says tpantazi branch and change it to master branch. But when we go here my wait command disappears and I want to see that wait occur.<\/li>\n<li>I want that wait so to fix this, I go back up to the branch at the top and from drop-down I select Create Pull Request. This will pull that branch back into our collaboration or master branch.<\/li>\n<li>This opens a new window back in Azure DevOps and it will set it up with a pull request. So, we set up the pull request from tpantazi into master and click Create and it will pop up for me to either approve or complete the pull request and merge.<\/li>\n<li>Back in Data Factory, once we refresh, we\u2019ll see that wait command come back into our master branch.<\/li>\n<\/ul>\n<p>Now you know how to connect an Azure Data Factory to an Azure DevOps repository. Preserving the code from a working version while you\u2019re making modifications is a great reason to use a DevOps repository.<\/p>\n<p><strong>Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or\u00a0 <a href=\"mailto:sales@3cloudsolutions.com\">sales@3cloudsolutions.com<\/a>.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Are you using Azure DevOps and want to know how to use it as a&mldr;<\/p>\n","protected":false},"author":35,"featured_media":9298,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[381,292,260],"tags":[385],"class_list":["post-10321","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-app-experience","category-app-innovation","category-data-ai","tag-devops","topics-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10321","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/35"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=10321"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10321\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/9298"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=10321"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=10321"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=10321"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}