Big Data Industry Report 2014

Click on Executive Summary, Table of Contents or IMEX Research for additional information on report.

Big Data – The Making of NextGen Big Companies

The internet has spawned an explosion in data growth in the form of data sets, called Big Data that are so large they are difficult to store, manage and analyze using traditional RDBMS which are tuned for Online Transaction Processing (OLTP) only. Not only is this new data heavily unstructured, voluminous, streams rapidly and is difficult to harness but more importantly, the cost of HW and SW infrastructure required to crunch it using traditional RDBMS, to derive any analytics or business intelligence from it online, is prohibitive.

To capitalize on the Big Data trend, a new breed of Big Data technologies (such as Hadoop and others) many companies have emerged which are leveraging new parallelized processing, commodity hardware, open source software and tools to capture and analyze these new data sets and provide a price/performance that is 10 times better than existing Database/Data Warehousing/Business Intelligence Systems. While most people would think of Google, Facebook as Media companies. In reality they are a myriad of other high growth internet oriented Big Data companies because the reality is that their businesses have been created due to their ability to effectively harness Big Data to their business advantage (e.g. Big Table from Google).

With the explosion in the use of the Internet, vast majority of data growth is coming in the form of data sets that are not well suited for traditional relational database vendors like Oracle. Not only is the data too unstructured and/or too voluminous for a traditional RDBMS, the software and hardware costs required to crunch through these new data sets using traditional RDBMS technology are prohibitive. To solve this, new companies have emerged that through using new Big Data technologies, are leveraging commodity hardware and open source software - from data capture, operational integration, advanced analytics to visualization.

Is Big Data trouble for the Existing Database Order?

IMEX Research believes that the economics inherent in using open source big data software running on commodity hardware will drive companies to consider these next generation systems when implementing new systems for new non transactional workloads. This is despite of being faced with the concomitant cost of developing expertise in the emerging alternatives. To illustrate this cost discrepancy in achieving high-performance solutions being provided today by traditional database vendors vs. the price points solutions put together using open source software and commodity hardware by the new generation big data vendors, three separate use cases illustrate the savings that are easily achieved as shown in the diagram as follows: (For details see chapter 3 of this report)

If RDBMS Could Handle Big Data Volumes, Why Bother?

IMEX Research believes that the economics inherent in using open source big data software running on commodity hardware will drive companies to consider these next generation systems when implementing new systems for new non transactional workloads, despite being faced with the cost of developing expertise in the emerging alternatives.

Leveraging the Competitive Advantages of Big Data

Much as Hadoop initially has been used by large web companies such as Google, Yahoo, Facebook for applications such as search engines, but its unrealized potential is huge.

IMEX Research believes that we are at a similar cross-road between SQL databases and upstart Hadoop solutions today to the 1970’s choice between these SQL databases and their predecessors. The success of one class of solutions vs. another ultimately will boil down to • The price of the chosen solution, and • The performance that can be achieved by a class of solutions.

To illustrate this cost discrepancy in achieving high-performance solutions being provided today by traditional database vendors vs. the price points solutions put together using open source software and commodity hardware by the new generation big data vendors, three separate use cases illustrate the savings that are easily achieved as shown in the diagram as follows: (For details see chapter 3 of this report)

Venture Investments Accelerated for Big Data

Hadoop isn’t the only thing going in big data, but it’s driving the bus at this point and it seems to have a Midas touch: everything that touches it turns to gold. Hadoop-based startups have raised $104.5 million since May 2011. This is over and above $159.7 million raised since 2009 till May 2011 when Cloudera closed its first round. These totals don’t include the recent or unattributed rounds for Odiago, high investment for Yahoo spinoff Hortonworks (now adopted by Microsoft). In addition the NoSQL in their focus on unstructured data, also have announced just more than $90 million in funding overall. Not counted in the above numbers are: - Opera solutions $84M (Silverlake, JGE, Accel, KKR, Tola Capital) - Infineta $15M (Rembrandt Venture Partners, Alloy Ventures, North Bridge Venture Partners) - Xignite $10M (StarVest Partners, Spring Mountain Capital) - Platfora $5.7M (Andreessen Horowitz, In-Q-Tel) Never mind, the many analytic database vendors that play in the big data arena that have been acquired over the past couple years for billions in aggregate.

Venture Investments Accelerated for Big Data

The Big Data 2011 Industry Report (updated in Feb 2012) brings into focus the key Database and Big Data technologies and their interplay, provides a perspectives on different players, who’s doing what to productize them and how they fit into the ecosystem. It profiles the growing number of companies — from startups like MapR to Cloudera, the present leader in the space leveraging Hadoop plus 104 others in full detail – both announced and in stealth, to the strategies being adopted by Relational Database/Data Warehousing/ Business Intelligence/Data Integration incumbents like Oracle, IBM, Microsoft, SAP, Teradata, SAS, Microstrategy etc. to embrace the emerging technologies. It outlines how new Big Data Infrastructure entrants the likes of EMC, NetApp, Cisco, Dell, Fujitsu, HP, Adobe and scores of others are planning new products to address this space.

It charts out the SWOT analysis of leaders and new suppliers as well as their competitive positioning and strategies. The report lists the various operational business intelligence challenges companies face and provides guidance as to how enterprise IT leaders can harness the deeper insights provided by Big Data to manage complex operational problems and empower their management in knowledgeable decision-making to get the competitive advantage for their companies.

For IT, the report lays out the rapidly evolving big data technology ecosystem - different big data technologies from Hadoop, Distributed File Systems, emerging NoSQL derivatives for implementation in private and hybrid cloud-based environments, Storage Infrastructure Requirements to Store, Access, Secure, Prepare for analytics and visualization of data while manipulating it rapidly to derive business intelligence online, to run businesses smartly.

For the Operational IT Management, the report delineates the operational issues businesses companies still encounter today in using legacy RDBMS systems despite their embracing fast access in-memory and solid state storage technologies. It details how IT is harnessing the emergent Big Data to manage massive amounts of data and new techniques such as parallelization and virtualization to solve complex problems in order to empower businesses with knowledgeable decision-making.

Further, the report outlines how the market opportunities in Big Data are crystallizing and picking up serious steam and to understand the challenges still hindering widespread adoption and where potential users can expect the market to go. It presents a 5 year market forecast 2010-15, market shares of leaders, likely M& A scenarios and examines go-to-market plans of leaders to provide big data solutions for several vertical industries such as financial services, healthcare and media. Finally it provides recommendation for vendors, channel players, end users and investors for timely play to leverage the opportunities presented by the emerging big data markets.

Click for Table of Contents For Overview of Industry Reports click here
or click the following icons