Success with big data

9 November 2017

Since McKinsey published its study entitled “Big Data, the next frontier for innovation, competition, and productivity” back in 2011, the concept of big data is now indispensable. Some 5 years later, the question is now to what extent companies and public-sector bodies have actually incorporated big data into their operations and what it has done for them. Are there now good examples of big data successes and what have companies and/or public-sector bodies done to achieve such successes? In other words: what are the success factors that will make big data “the new oil”?

It is extraordinary that in July 2016 the Gartner hype cycle already no longer includes the term “Big Data”. Although it does include all kinds of other developments directly related to big data. According to Gartner, developments such as Predictive Analytics, Data Driven Marketing, Augmented Reality Marketing and Multidimensional Campaign Attribution are at the peak of their life cycle. You can assume that big data is now taken for granted in virtually every organisation.


Big Data in Practice

So I am in search of the many successes. When you search online, you will soon come across Bernard Marr’s book published in 2016: “Big Data in Practice”. The subtitle gives you high expectations: “how 45 successful companies used big data analytics to deliver extraordinary results”. Personally I would have written that in the present tense, because I don’t have the impression that organisations have stopped cashing in on their big data yet. Grammar doesn’t seem to be Marr’s strong point.

The book provides a summary of 45 case studies in which the organisations’ success (Narrative Science and IBM Watson are also considered organisations) is attributed to their utilisation of big data. Bernard Marr applies a fixed pattern with each case study:

  • What problem is big data helping to solve?
  • How is big data used in practice?
  • What were the results?
  • What data was used?
  • What are the technical details?
  • Any challenges that had to be overcome?
  • What are the key learning points and takeaways?

The nice thing about the book is how varied it is. The examples aren’t just about large multinationals, but small and non-profit organisations too. It shows that big data is already being used in many industries. There are of course the leading companies of our time where big data is an intrinsic part of their model, e.g. Walmart, Google, Facebook, Twitter Apple, Microsoft, Netflix, LinkedIn, Airbnb and Uber. But what about Rolls Royce, Royal Bank of Scotland, the US Olympic Cycling Team, London Zoo, Caesars Casino, Walt Disney Parks and Resorts and Dicky’s Barbecue Pit. Certainly there are a lot case studies that make you think: I can think of a few more like that.

In virtually all cases, they have large volumes of data but only rarely unstructured and/or streaming data. One striking part was the summary of the technologies that organisations use to store, analyse and apply their big data: Apache, Apixio, Azure, BigQuery, Cassandra, Cloudera, Conjecture, Datameer, DeepQZ, DMX, EC2, EMC, Flume, Fusion, Hadoop, HANA, HDFS, Hive, Java, Lambda, Mahoot, MapR, MK:Smart, Mongo, Oozie, Predix, Presto, Python, R, Redshift, Solr, Spark , Splunk, SQL MemSQL MySQL and MySQLSSD, Sqoop, TeraData, V-Block, Vertica, Voldemort, Yellowfin. What happened to SAS and SPSS?

The description of the case study and the proof of the claim that the organisations would achieve extraordinary results could have been done rather more substantially for almost every case study. Marr sometimes hides behind the fact that company secrets are involved. In almost every case, it is difficult to work out exactly what data the organisations are using and how and what the actual results are.

When it comes to the challenges that the big data organisations still face, in many cases he cites the shortage of good data analysts and/or data scientists. This also applies to big data organisations, despite the fact that they use artificial intelligence and machine learning on an enormous scale.


Other examples

Continuing the search for big data successes does not deliver a great deal of specific cases. The large data companies cited by Marr is frequently used as examples. Only IBM provides other examples, all from their own circle of clients and with their own technology. A study by Forbes in late 2015 reportedly demonstrated that only 15% of Fortune 500 companies successfully apply big data analytics to make their business more efficient and more effective.


My conclusion

My search leads me to the conclusion that organisations that are good at turning data into information and applied knowledge have an advantage over organisations that don’t do it or only do it to a limited extent. But Thomas Davenport already proved that back in 2007 with his book “Competing on Analytics”. And we have been proving that in our daily practical experience ever since we were founded in 2002. According to the literal definition of big data, its application can make all the difference in exceptional situations. However, many organisations would do well to start making the data they have into something that can be applied. In spite of all the modern technologies for data analysis, this remains a specialist skill and good analysts tend to make more of a difference than adding even more data.

Latest news

In 2022, we are going to have a great year of celebration! Many weddings expected

21 February 2022

Plan your calendar free and make sure you have plenty of party clothes in your closet,... read more

Building a book recommender from scratch

1 April 2021

Almost every day we go online we encounter recommender systems; if you are listening to your... read more

Michelin stars 2020 announced– how accurate was our prediction?

13 January 2020

Today was the moment of truth for many restaurant owners, and for us as well. The... read more

Subscribe to our newsletter

Never miss anything in the field of advanced analytics, data science and its application within organizations!