{"id":16107,"date":"2014-07-08T11:40:00","date_gmt":"2014-07-08T18:40:00","guid":{"rendered":"https:\/\/devwww.3cloudsolutions.com\/post\/hadoops-momentum-is-unstoppable-2\/"},"modified":"2024-01-04T10:33:55","modified_gmt":"2024-01-04T18:33:55","slug":"hadoops-momentum-is-unstoppable","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/hadoops-momentum-is-unstoppable\/","title":{"rendered":"Hadoop’s Momentum is Unstoppable"},"content":{"rendered":"
\n

\"PastEvery new technology has a cycle of introduction, awareness, evaluation and–if the technology is relevant–eventual mainstream acceptance and adoption. \u00a0Gartner has even coined a term for it: the hype cycle<\/a>.<\/p>\n

<\/p>\n

Hadoop’s Meteoric Rise<\/h2>\n

Hadoop technology is working through its acceptance cycle too. What’s been stunning, though, is to watch how Hadoop technologies have moved so quickly through the cycle. \u00a0Its progress in finding effective use cases and proving its worth has been nothing short of stunning. Its momentum seems–as Forrester calls it in its Q1\/2014 Forrester Wave analysis (downloadable from HortonWorks)<\/a>:”Unstoppable”.<\/p>\n

As part of our Strategy & Architecture Services<\/a>, BlueGranite frequently consults with key clients to understand what technologies are on their radar and which are the most relevant to them. Not two years ago when we spoke with clients about Hadoop<\/a>, most viewed it as a curiosity, and possibly an over-hyped technology that would be soon exposed as one not relevant to “real” enterprise customers.<\/p>\n

\"Launch<\/p>\n

How things have changed! \u00a0Today about half our enterprise customers have deployed Hadoop. Some with the help of our design and implementation teams, others on their own. \u00a0For the majority, deployments are still within the scope of evaluation and organizational learning. \u00a0But we’re starting to see production deployments for new data processing and analytical uses cases–and the results are compelling.<\/p>\n

Why Hadoop will evolve much faster than Linux<\/h2>\n

A short few years ago, the best thing Hadoop had going for it was its open source<\/a> distribution model that can<\/em> bring large-scale data processing capabilties free of licensing cost and exotic hardware. \u00a0Yet–at that time–the trade-off to enjoying this low-cost platform was to give up the comfort of enterprise-grade vendor support and product stability.<\/p>\n

I clearly recall not long ago listening to Hadoop thought leaders suggesting that only large-scale tech companies could support Hadoop because only they could hire Stanford PhDs that were required to patch and recompile the Apache code<\/a>. \u00a0With the rock-solid backstop of BlueGranite’s Hadoop distribution partners,\u00a0HortonWorks<\/a>\u00a0and\u00a0Microsoft<\/a>, that line of thinking today would be laughable. The vast majority of enterprise customers will only think about Hadoop as a vendor-provided software that’s installed (not open source software that’s compiled on-site).<\/p>\n

\"TimeEvolvesTechnology\"The situation was not unlike\u00a0the early days of Linux<\/a>, when using the free operating system meant needing to add staff with the ability to provide self-support for low-level operating system malfunctions.\u00a0But Linux matured, distribution vendors like Red Hat<\/a> and SuSE<\/a> made Linux a supportable platform enterprise customers could trust. A healthy ecosystem of systems integrators and consultants to fill knowledge gaps developed to further reduce the risk and cost of Linux implementation for enterprises.<\/p>\n

The same is happening with Hadoop. But this time, a successful distribution\/support model<\/a> pulled directly from the playbook of\u00a0Linux vendors and free distributions<\/a> was quickly adopted. Add to that an unprecedented level of venture capital and corporate sponsorship that funds the development of the platform, and a sometimes mystifying level of collaboration by competitors on the technology itself. The result is progress at a break-neck pace.\u00a0Today Hadoop has already evolved into a data platform whose capabilities–in many ways–rival the world’s best proprietary data platforms.<\/p>\n

Is Hadoop for Everyone?<\/h2>\n

Whether Hadoop is for Everyone is certainly a loaded question. \u00a0Of course not! \u00a0No single technology is a silver bullet for everyone. \u00a0There are organizations who will never need Hadoop, just as there are those who will never need an intranet portal or a cloud strategy.<\/p>\n

But for most enterprises (and midsize companies that have all the trappings of enterprises but at a smaller scale), the answer is “Yes”.<\/p>\n

Hadoop will soon be about much more than Gartner’s the three V’s model (Volume, Velocity, Variability of data)<\/a>. \u00a0It’s evolving into a data processing platform that will be better and less expensive than what most enterprises use today. \u00a0Yarn<\/a> gives it resource management (control over shared computing power allocated to various workloads). \u00a0Hive<\/a>, Impala<\/a> and Spark<\/a> are beginning to turbocharge SQL performance against data stored in Hadoop<\/a>. And the bottom line: TCO. Hadoop provides Saks Fifth Avenue level MPP (Massively Parallel Processing)<\/a> technology at WalMart prices anyone can afford.<\/p>\n

Even deploying Hadoop at small scale to address specific workloads is compelling. \u00a0For example, I recently posted an article on my personal blog<\/a> showing how to use Hadoop to collect Twitter data<\/a> containing a set of keywords. Sure, you can probably figure out how to do this in Excel. \u00a0But how scalable is that? It’s not. \u00a0Try it yourself<\/a>: it takes maybe a half hour to get up and running on a Hadoop cluster. \u00a0Don’t have one? Use a free, pre-configured, single-server Sandbox VM<\/a>. \u00a0Value? High. Cost? Free. Easy? Yes. Stanford PhD required? Definitely not.<\/p>\n

We live in interesting times<\/h2>\n

In technology, every few years we seem to be on the cusp of a pivotal moment. \u00a0Moving from inflexible, not-very-scalable ISAM databases<\/a> to RDBMS technologies<\/a> was pivotal. \u00a0Moving from Client\/Server<\/a> to multi-tiered software architectures<\/a>\u00a0was pivotal. Moving to the cloud<\/a> is definitely pivotal.<\/p>\n

Putting massively parallel processing (MPP) technology<\/a> in the hands of every organization <\/em>will prove to be equally pivotal. \u00a0It’s not less significant than how the Internet has leveled the playing field, giving small Indie Developers<\/a> equal marketing opportunities to Fortune 1000 companies.<\/p>\n

Yes, we live in very<\/em> interesting times.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Every new technology has a cycle of introduction, awareness, evaluation and–if the technology is relevant–eventual mainstream acceptance.<\/p>\n","protected":false},"author":21,"featured_media":15020,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[260],"tags":[304],"class_list":["post-16107","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-ai","tag-modern-data-platform","topics-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/16107","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/comments?post=16107"}],"version-history":[{"count":0,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/posts\/16107\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media\/15020"}],"wp:attachment":[{"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/media?parent=16107"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/categories?post=16107"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/3cloudsolutions.com\/wp-json\/wp\/v2\/tags?post=16107"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}