Add to Technorati Favorites

 Subscribe in a reader

Trends and Outliers

TIBCO Spotfire's Business Intelligence Blog


Big Data and the Call for Evidence-based Management

big data opportunities and challenges Big Data and the Call for Evidence based ManagementMany discussions around big data are often centered on the sheer volume of the information involved, including the massive quantities of unstructured data that’s being generated in social media and other channels. This includes text, images, emails, tweets, videos, voice messages and other information that people share on Facebook, Twitter, blogs, forums, etc.

Unstructured data is expanding exponentially. As John Myers (@johnlmyers44) notes in a recent Enterprise Management blog, “often times what is missed are the analysis goals of the big-data environment.” In other words, not seeing the forest for the trees.

In his blog, Myers points to a recent article by Don Peppers and Martha Rogers, Ph.D., founding partners of Peppers & Rogers Group, a customer management consultancy. In their article, Peppers and Rogers talk about the importance of focusing first on facts and analysis and not becoming overwhelmed by the flood of data that’s available to us.

Peppers and Rogers point to evidence-based medicine for guidance. As Peppers describes in a July 2011 Fast Company blog, evidence-based medicine involves the use of statistical studies and quantified research by physicians to form diagnoses. The value of relying on the statistical findings, writes Peppers, is that it enables doctors to make decisions not only based on their own perceptions but also from the best available evidence.

‘The reason doctors are encouraged to look at the statistical evidence before rendering their own judgments is because of the confirmation bias, our natural human tendency to place more credence in whatever facts or numbers confirm the point of view we already have,” says Peppers.

Likewise, in order to deal with the crush of unstructured data that’s now available, business leaders will also “need to put aside their pre-conceived notions and rely more on what the numbers actually say,” Peppers adds.

For his part, Myers agrees with the evidence-based management approach of Peppers and Rogers. He even suggests taking it a step further. Myers asserts that “any successful big-data analytical environment needs to focus on how the data can impact a company’s top line, bottom line or both when it is conceived and constructed rather than focusing on how many petabytes of information will be stored.”

Indeed, big data is best utilized when companies focus on how the information can affect their businesses or how it can be applied to improving business performance and not when the focal point is placed on the volume or structure of the data.

Print post

Is your data hiding something?
Download Your Free 30-Day Evaluation of TIBCO Spotfire®

Filed under: Big Data



Evidence based medicine regards phisicochemical behaviour of human body, which is accountable and thus may conclude in statistical findings. A company top and bottom line are conditioned by multiple factors and if not all probably most of them inpredictable and not subject to statistical conclusions, like human behaviour itself. Here is in my opinion the main driver for example of marketing failure: giving statistics the central place that should be occupied by creativity and innovation.

david k waltz

McKinsey released a series of “Big Data” articles, and one of the points in them was that management processes may need to change.

I thought about that when reading the “confirmation bias” phrase, and also generally when thinking about “evidence-based management”.

Often, the evidence is selected by those with a particular agenda within the organization – get their project approved, increase their budget, etc.

This will be a challenging element to implementing any “big data” processes, as the winners in the old system will need to voluntarily give up. That does not seem realistic.


One Trackback

[...] data analytics community apparently appreciates “evidence-based” data analytics. This October goodie from the Spotfire blog gives some insights on how we have to “focus first on facts and analysis [...]


Post a Comment

Your email is never shared. Required fields are marked