Analysis: While technologies like Spark continue to interest the maturity of the big data space means the messaging must change.
Businesses have now entered an era where data is vital to success, where differentiating on how they use data could be the path to the top of the market and where failure to do so, could see the demise of the business all-together.
Big data’s impact on businesses has resulted in the creation of new C-level roles and posed the challenge of developing skills to be able to keep up with the demand for data analytic processing and the tools that have swamped the market.
While conferences are typically swamped with updates, enhancements, and the odd new product, the Strata & Hadoop Conference in London recently had a greater focus on strategy than it did technology.
To focus on data strategy is a sensible place for many of the tech vendors in the Hadoop ecosystem to go, after all, making money from consultancy style services would add another source of revenue in addition to the software, training and certification revenues which they already receive.
In the build up to the event, CBR talked to several vendors that have significant plays in Hadoop – Cloudera, Informatica, Trifacta, Hortonworks, and Cray. While all to some extent had their own product upgrades or new releases recently available, the theme of the conversations was predominantly around strategy.
Having an effective data strategy is clearly extremely important, without one a business is likely to fail in their strategy and with new regulatory requirements such as EU General Data Protection Regulation on the horizon, now seems to be an extremely apt time for vendors to be advising on how to get strategy right.
Clarke Patterson, senior director, product marketing, Cloudera told CBR that over the next several months the company would be talking more about a "journey to success" by helping businesses understand the people, process, and technology.
Patterson said: "I need to kind of peel apart the layers and understand which projects within It I am specifically going to attack first, which projects within Hadoop specifically I should be thinking about and then applying those to different departments and different strategic initiatives within the business."
This will help a business to figure out what businesses processes should be dealt with first, addressing whether there is a skills gap, and figuring out how to remedy that.
Patterson admitted that it isn’t really a ground breaking concept but that the company had received a lot of feedback from customers about the difficulties posed by jumping in too deep with Hadoop.
Mixed in with this messaging is the placement of Cloudera’s technology as an integral component, something that is mirrored by all the vendors I spoke to.
Informatica mirrored much of what Cloudera said, identifying the importance of people in addition to process and technology.
Greg Hanson, VP business operations, EMEA, Informatica, told CBR: "One of the things we’ve always said, in order to be successful in any project you need to have a combination people process and technology, obviously we have all the technology but it’s also about the people."
Hanson believes that the world of data is changing, heading from ‘Data 1.0’ where data is used to automate tasks, to ‘Data 2.0’ where it is used for enterprise projects, to ‘Data 3.0’ where businesses understand that data is vital to success.
With data becoming vital to success, businesses have responded by creating new roles, the Chief Data Officer, and data stewards to name two.
"I think for the first time what we are seeing is that these roles exist at the C-level and that’s important because now they are starting to exist at a board level they can really start being the agent of change and make that move in organisations to data 3.0," said Hanson.
Like Cloudera, Informatica has its own path to success with a five level approach called data management strategy.
With any reasonably new technology advancement there will be those that are quick to adapt and change and those that drag their heels. Hanson said that when it comes to appointing well defined CDO roles, highly regulated areas such as financial services and pharmaceuticals are leading the way.
The change in focus to a strategy based one highlights the maturity of the big data market but it also suggests that innovation in the area may be slowing.
Gartner places big data in its ‘trough of disillusionment’ and while innovation is still taking place, there appears to be less of it, forcing vendors to change tact.
Increasingly a few technologies in the Hadoop ecosystem like Spark are receiving the majority of the attention, highlighted by Microsoft making a serious commitment to the processing engine last week.
The slowdown in innovation is not limited to the big data world as other areas of the tech industry also struggle to continue to maintain the hype around the latest and greatest breakthroughs.
This isn’t necessarily a bad thing.
For a start, it allows businesses to catch up. If businesses are expected to go through a digital transformation but the tech industry continuously brings out technology that replaces the old, then it is almost an insurmountable task.
Businesses require time in order to transition to new technologies and tech companies giving them time to focus on strategy and getting it right, should result in a better long term result.