{"id":19013,"date":"2023-02-04T18:23:02","date_gmt":"2023-02-05T02:23:02","guid":{"rendered":"https:\/\/threecloud.wpengine.com\/?p=19013"},"modified":"2023-06-01T14:23:43","modified_gmt":"2023-06-01T21:23:43","slug":"3-simple-steps-to-data-ingestion-using-partitioning","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/3-simple-steps-to-data-ingestion-using-partitioning\/","title":{"rendered":"3 Simple Steps to Data Ingestion Using Partitioning"},"content":{"rendered":"<table style=\"border-collapse: separate; width: 100%; border-spacing: 20px;\">\n<tbody>\n<tr>\n<td style=\"width: 50%;\"><a href=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2023\/05\/3-Steps-to-Data-Ingestion.pdf\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-18994\" src=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-17-at-3.05.02-PM.png\" alt=\"3 Simple Steps to Data Ingestion Using Partitioning\" width=\"400\" height=\"515\" \/><\/a><\/td>\n<td style=\"width: 50%;\">When you have large data sets with hundreds of millions or billions of records, ingesting that data efficiently can be a challenging feat. In many cases, these data sets may be loaded on a nightly schedule. Your nightly data ingestion pipeline must be complete within a certain time frame to avoid negatively impacting your downstream processes or analyses. So, what can you do when a job you are running each night grows too large to be processed in your time frame? How can you better manage the ingestion of these large datasets?<\/p>\n<p><a href=\"https:\/\/3cloudsolutions.com\/wp-content\/uploads\/2023\/05\/3-Steps-to-Data-Ingestion.pdf\" target=\"\" rel=\"noopener\">3 Steps To Data Ingestion<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"excerpt":{"rendered":"<p>When you have large data sets with hundreds of millions or billions of records, ingesting&mldr;<\/p>\n","protected":false},"author":21,"featured_media":10852,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[260,297],"tags":[],"class_list":["post-19013","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-ai","category-data-platform","topics-tipsheet"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/19013","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=19013"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/19013\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/10852"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=19013"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=19013"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=19013"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}