{"id":10322,"date":"2020-08-20T00:00:00","date_gmt":"2020-08-20T05:00:00","guid":{"rendered":"https:\/\/threecloud.wpengine.com\/post\/how-to-connect-azure-databricks-to-an-azure-storage-account\/"},"modified":"2022-11-30T09:24:38","modified_gmt":"2022-11-30T15:24:38","slug":"how-to-connect-azure-databricks-to-an-azure-storage-account","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/how-to-connect-azure-databricks-to-an-azure-storage-account\/","title":{"rendered":"How to Connect Azure Databricks to an Azure Storage Account"},"content":{"rendered":"<p>In continuation with our Azure Every Day <strong>mini-series on Azure Databricks<\/strong>, I will be covering some key topics within Databricks such as Azure Key Vault, storage accounts, PowerPoint and DevOps. If you\u2019re just starting out with Databricks, you may want to check out our previous posts on <a href=\"\/azure-databricks-101\" target=\"_blank\" rel=\"noopener noreferrer\">Databricks 101<\/a> and <a href=\"\/getting-started-with-azure-databricks\" target=\"_blank\" rel=\"noopener noreferrer\">Getting Started with Azure Databricks<\/a>. <strong>Today\u2019s post is focused on accessing Azure Storage accounts<\/strong>.<\/p>\n<p><strong>Azure Databricks connects easily with Azure Storage accounts using blob storage. To do this we\u2019ll need a shared access signature (SAS) token, a storage account, and a container.<\/strong> We can peruse our files with the downloadable application called Azure Storage Explorer.<\/p>\n<p><strong>My video included below is a demo of this process. Here\u2019s how to connect Azure Databricks to an Azure Storage Account (blob storage):<\/strong><\/p>\n<ul>\n<li>With Azure Databricks loaded, we click on Launch Workspace which takes us to our Azure infrastructure.<\/li>\n<li>In my demo, I already have a cluster up and running and a notebook. A notebook is how we do our coding within Databricks.<\/li>\n<li>First thing we need to do is create a storage account, in my case I created a blob storage. I also created a container that I named demo.<\/li>\n<li>I can access that container through the Azure Storage Explorer. I did this by connecting to my Azure Storage account and drilled into my blob storage and my demo container, then uploaded 2 files which are CSV in nature.<\/li>\n<li>You need to be aware that within Azure Databricks you\u2019ll use some Python code to connect to your storage account. (You can see this code and detail about it in my video.)<\/li>\n<li>There are also some steps you\u2019ll need to follow to generate an SAS connection string which you can also see in my demo.<\/li>\n<li>Once we\u2019ve done that, we go back to our Databricks and click Ctrl\/Enter. Since the cluster is already up and running, it\u2019s going to send the command to the cluster and the cluster will perform the action.<\/li>\n<li>If it\u2019s successful, we\u2019ll have a connection. We will have a link and we\u2019ll mount the demo folder. It may take a few minutes to run and you\u2019ll see if it succeeded, then you should be able to see your files.<\/li>\n<li>If it succeeded, we now have a mounted folder structure in our blob storage account. We can display the contents of this folder and I see that I have my 2 CSV files in there.<\/li>\n<li>That is how easy it is to connect your blob storage in Azure in Azure Databricks using your connection.<\/li>\n<\/ul>\n<div class=\"hs-embed-wrapper\" style=\"position: relative; overflow: hidden; width: 100%; height: auto; padding: 0; max-width: 560px; max-height: 315px; min-width: 256px; display: block; margin: auto;\" data-service=\"youtube\" data-responsive=\"true\">\n<div class=\"hs-embed-content-wrapper\">\n<div><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/3pNJX0Jd9pg\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/div>\n<div style=\"position: relative; overflow: hidden; max-width: 100%; padding-bottom: 56.25%; margin: 0px;\">Keep an eye on this blog for more on our mini-series on Azure Databricks. Talk to us if you\u2019d like to learn more about how to leverage Databricks, Azure in general or any Azure product or service.<\/div>\n<\/div>\n<\/div>\n<p><strong>Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or\u00a0 <a href=\"mailto:sales@3cloudsolutions.com\">sales@3cloudsolutions.com<\/a>.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In continuation with our Azure Every Day mini-series on Azure Databricks, I will be covering&mldr;<\/p>\n","protected":false},"author":29,"featured_media":10685,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[260],"tags":[],"class_list":["post-10322","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-ai","topics-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10322","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=10322"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/10322\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/10685"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=10322"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=10322"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=10322"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}