Add to Technorati Favorites

 Subscribe in a reader

Trends and Outliers

TIBCO Spotfire's Business Intelligence Blog


A Cautious Approach to Big Data

big data opportunities and challenges A Cautious Approach to Big DataBig data can provide companies with powerful insights about everything from customer behaviors and preferences to new product opportunities. As cited in a recent article featured in MIT’s Technology Review, big data is used by companies such as Netflix to predict customer behavior, recommend products and deliver social media ads. But as the Technology Review article also mentions, big data can “have dangerous results.”

For instance, a new paper cited by Technology Review, entitled “Six Provocations for Big Data,” presented at a recent Symposium on the Dynamics of the Internet and Society, lists some of the reasons that businesses and other organizations should take a cautious approach to using big data. While consumer privacy issues are among the most evident issues decision makers should concern themselves with, big data can also provide organizations with distorted or incomplete views of customer and market trends.

Kate Crawford, an associate professor at the University of New South Wales, who co-authored the “Six Provocations” paper with Microsoft senior researcher Danah Boyd, writes that big data doesn’t always offer accurate predictions about people’s behaviors. This is partly true when organizations over rely on information about people’s behaviors as expressed in social channels such as Facebook or Twitter.

Part of the problem here is that comments that people post in social media don’t provide companies with complete views of their behaviors, attitudes, or preferences, Crawford tells Technology Review. Says Crawford, “Facebook is not the world.” Nor is Twitter. Research that Crawford and Boyd conducted reveals that while tweets are commonly analyzed by companies to understand and act on people’s moods and attitudes about politics and other aspects of life, roughly 40% of people who sign onto Twitter do so only to listen and not to post. This suggests that posts come from certain types of people and don’t represent a more random sample, according to Crawford and Boyd.

Leveraging a Wider Range of Data Sets

Companies can benefit greatly from the use of big data, analytics, and predictive algorithms to help predict future customer behavior and to spot other business trends. However, as Crawford notes, data that’s shared by people in social media doesn’t provide companies with complete views of their customers. Indeed, companies can benefit from a wider range of data sets that’s available to them by analyzing data that customers share through the multiple channels they use to interact with companies, including recorded contact center calls, email, online and mobile. But companies shouldn’t stop there. A growing number of companies are beginning to blend insights from customer interaction data along with transactional and operational information.

Customers share a lot about themselves in social networks and through other interactions they have with companies – both deliberately and unintentionally. But it’s important for analysts and decision makers to recognize that people don’t always communicate their full intentions or behaviors in their social network comments or even through customer satisfaction and other types of surveys they fill out. Companies that cast a wide net are more likely to haul in multidimensional insights that can ultimately help executives make more informed decisions.

Next Steps

If you want to learn more about the tools needed to predict future customer behavior, check out this pre-recorded webcast.

Print post

Is your data hiding something?
Download Your Free 30-Day Evaluation of TIBCO Spotfire®


david k waltz

Big data has gotten a lot of hype recently (and I admit I have helped that along occasionally in my blog) but, being of analytical nature, I had a lingering unease, thinking along the lines of “yeah, but what’s the other side?”. Thanks for helping to answer that question.

One of my takeaways from the twitter/social media paragraph is “so it is not a representative sample”, which is a basic statistical concept, yet apparently people are out there concluding things on the basis of this.

There has been a lot of back and forth on various LinkedIn groups about what qualifies someone to be perform “Analytics”, and there is a camp that statistics is inadequate to the job and not necessary. I am not of that mind myself, but it is nice to find examples that add evidence to the discussions.


I thought that was surprising as well. It indicates some readers feel strong affinity and trust for their online tribes of fellow readers.
Customer Satisfaction Survey Questions


Post a Comment

Your email is never shared. Required fields are marked