Add to Technorati Favorites

 Subscribe in a reader

Trends and Outliers

TIBCO Spotfire's Business Intelligence Blog

Monthly Archives: May 2012


Does It Take a Data Scientist to Find Gold in Big Data?

Any data scientist knows that the true value of data comes from its refinement.  In a recent CNET article, Alex Yoder compared data to the value of other can’t-live-without assets – oil and gold. However, he says that big data has a major differentiator – “data has no intrinsic value.

big data gold 150x150 Does It Take a Data Scientist to Find Gold in Big Data?Yoder (@yodera) writes, “Gold requires mining and processing before it fines its way into our jewelry, electronics and even the Fort Knox vault. Oil requires extraction and refinement before it becomes the gasoline that fuels our vehicles. Likewise, data requires collection, mining and, finally, analysis before we can realize its true value for businesses, governments and individuals alike.”

What’s interesting about this comparison is that behind the mining, processing and analysis is big science. It takes a scientific approach and processes to find the value in data and precious commodities. Yoder defines the analysis portion of extracting value from data as the science.

The Science of Finding Gold in Data

As Yoder points out, there’s also science in the mining and processing of the data as well – just go back to our interview with Gregory Piatetsky-Shapiro (@KDNuggets) on the topic of data science and predictive analytics.

Piatetsky-Shapiro says, “What we want to find in data is some understandable knowledge and not just incomprehensible patterns. He adds, “[A] better understanding of predictive models will contribute to increased trust for such models.”

Piatetsky-Shapiro is a living example of how science can extract value from data. He’s developed models to help change the world. In our interview, he told us he has developed models for life-changing predictive analytics such as predicting child support non-payment and attrition to predicting drug effectiveness to developing methods for identifying fraud in online auctions.

Didn’t Gold Mining Start with a Simple Quest for Something More?

While Yoder and Piatetsky-Shapiro both point to the methods, models and software as keys to refining value, they also recognize the human element. Yoder says, “It takes complex algorithms, powerful computing and perhaps most of all, human analysts to build and administer the big science that turns the ‘then and now’ nature of big data into ‘when.’”

He points to a projection from McKinsey Global that the US needs 140,000 to 190,000 additional deep analytical minds working to derive the value of data. Often labeled as data scientists (quite fitting with the topic of big data being backed by big science), Yoder says these experts are thought to be the next class of “digital and corporate geniuses.”

Piatetsky-Shapiro’s take is that the data scientist role is much like an engineer or wizard with a skill set that combines “code tuning, business insight and knowing how to extract business value from data.”

Our Take: Give Your People Gold Fever

We think there’s a place and a definite need for these data wranglers. However, if an organization is going to join the gold rush that big data promises, don’t you think we have to start with the users? And give them the tools they need – the picks, the pans and the drive or “gold fever” to “head for the hills?”

Their tools are analytics systems (maps) and the “quest for knowledge” that allows them to inspect the data landscape and find the gold nugget-filled streams and hills.

Mark Lorion (@mark_lorion) alludes to why the tools and the questions matter most in a recent interview on DM radio. He says, “If you measure it, you’ll change it. What’s hiding in your data that you haven’t measured?”

For example, he says when a business user has questions, the tools often get in the way of the answers. He says it’s hard for a user to find the right answers in a spreadsheet or other tools when the presentation is hiding what he’s trying to see. It’s frustrating because the user knows he can beat the competition if he can get reliable information from the data.

Maybe the gold or “knowledge discovery” (as data scientist Piatetsky-Shapiro says) starts with the right tools and the right questions and a little bit of fever.

Next Steps: See how Spotfire version 4.5 empowers users to discover actionable insights hidden in big data and unstructured information in our on-demand webcast, “What’s New with Spotfire 4.5.”

Amanda Brandon
Spotfire Blogging Team

Print post


Top Five Ways to Advance Agile BI

spped2 150x150 Top Five Ways to Advance Agile BIThe need for management to receive timely information has steadily increased during the last few years, according to a study conducted by the Aberdeen Group.

As the trend continues, IT departments will be subject to increasingly tighter decision-making windows and they’ll struggle to deliver data with greater speed and increased accuracy.

Referencing a compilation of studies, the Aberdeen Group reports that 42% of decision makers participating in the agile business intelligence research need data within one day of the event, 10% within the hour and 18% within minutes.

Continue reading »

Print post


From Transaction to Customer Engagement with Business Analytics

Online Customer Experience 150x150 From Transaction to Customer Engagement with Business AnalyticsMuch of the promise of big data lies not in the ever-increasing volume of information available to organizations, but in the insight – notably with regard to customers – that lies buried within these large data sets.

Customer intelligence – including the sentiment behind consumers’ buying choices and the types of product offers that may appeal to them – can only be gleaned by moving beyond  the traditional consumer “transaction” to using business analytics to delve into consumer-generated content across the social Web and boost the bottom line by focusing on the overall customer experience across all channels.

Continue reading »

Print post


The Data Analytics of Gas vs. Charcoal Grills Revisited

Last year, Mark Lorion (@mark_lorion), our vice president of marketing, took a quick break from the exciting world of data analytics to give us his answer to a burning question – charcoal or gas grill – just in time for Memorial Day. See his data visualization from last May here.

While we’ve never solved the dilemma of which is better (we think it has more to do with a personal preference), we’re firm believers in asking questions to find answers in the data. So, we went to a trusted source for some new facts surrounding the controversy of coals versus blue flames – Google.

Here’s what we found:

The Huffington Post has published a poll that lets you note your preference, read some comments from their Facebook fans and then decide if you still agree with your initial answer. It’s a nice data collection move!

Food columnist Jim Hillibish “bursts open the urban myths of grilling” in this column. A few interesting stats:

  • The grilled taste comes from the fat dripping and burning on the heat source, so it has more to do with the fat content of your meat than the fuel source;
  • The “smoking” of meat takes about five minutes of contact with the smoke on a grill;
  • If you were to use pine chips to smoke your ribs or pork roast, you’d taste something equivalent to turpentine. Fruit tree wood is better.
Finally, for all the data pros out there stuck at work today, here’s a little meat on the current trends in grilling to get you primed for the long weekend!
beef data points infographic  04 09 12 copy1 The Data Analytics of Gas vs. Charcoal Grills Revisited
Next Steps: Tweet us your plans for the unofficial start of summer and tell us what you think about the charcoal vs. gas grill debate.



Print post


Getting That Extra Bang for Your Business Analytics Bucks

analytics roi1 Getting That Extra Bang for Your Business Analytics BucksWhen it comes to your organization’s use of business analytics to maximize business performance, is your company getting the full bang for its investment bucks?

A new study from Nucleus Research gives decision makers even more to consider regarding the returns they’re seeing from their analytics efforts.

According to the study, as business analytics solutions mature and become more refined, the ROI from investing in these tools rises substantially. In its study of 60 deployments, Nucleus evaluates the returns that companies are seeing from investments in business intelligence, performance management, predictive analytics and supporting data management technologies.

Continue reading »

Print post


Enterprise Data Mashup: Taking Data Integration to the Next Level

“Enterprise Data Mashup” is the new mantra when analyzing a customers complete experience with your brand.

According to a new Gartner Inc. study, customer experience is now a top 10 priority for CIOs. datalevel 150x127 Enterprise Data Mashup: Taking Data Integration to the Next LevelStill, in order for companies to become more customer-centric, they need to gather and act on the full range of information that customers share across the multiple channels they use to interact with companies, including web, mobile, social, chat, email, and voice.

Data silos constrain an organization’s agility, place limits on efficiency, and prevent decision makers from gaining a 360-degree view of customers, including their behaviors, needs, and attitudes  According to research conducted by Kyield, data silos result in $1.2 billion in preventable costs for the pharmaceutical industry and more than $100 billion in lost opportunities annually for the federal government.

Of course, there’s a distinct difference between simply collating data streams from different channels and sources and ensuring that the right mix of information is being collected to enable decision makers to gain real-time views of customer and market trends.

Part of the challenge faced by many companies is the need to address data latency to ensure that the most timely and important information is being blended together so that decision makers don’t miss out on new market opportunities as they arise.

A great starting point for achieving data integration begins at the C-level. In order for companies to successfully share and blend meaningful data from a variety of sources, data integration efforts need to be championed by a senior executive who is able to effectively communicate and extol the benefits for doing so.

This is different than someone who might be labeled a “data champion” – typically an IT professional who is knowledgeable about the business and can be positioned as an effective liaison to business leaders.

Line of business and functional leaders are often reluctant to share customer data housed by their divisions with other parts of the business. That’s why it’s critical for a senior corporate executive (CEO, COO) to stand behind the benefits of data integration. This includes demonstrating the business benefits that can occur such as small wins that reflect an uplift in cross-sell or upsell opportunities or other business performance gains.

The right BI and analytics tools can make it a lot easier for companies to create an Enterprise Data Mashup, thus minimizing the amount of effort needed to install APIs or middleware between various data systems.

It’s the companies that break through the organizational boundaries that keep them from acting on all the available data that will ultimately succeed.

Next steps:

  • Check out our complimentary “5-Minute Guide to Business Analytics” to learn how user-driven “analytic” or “data discovery” technologies help business and technology users more quickly uncover insights and speed action.
  • See how Spotfire version 4.5 empowers users to discover actionable insights hidden in big data and unstructured information in our on-demand webcast, “What’s New with Spotfire 4.5.”

Print post


Data Discovery: 6 Questions to Ask for Real Insight From Big Data

When considering your data discovery options for gaining real business insight, big data is a big deal.

The volume of data from social media, consumer online behavior and the cloud is streaming into companies at an ever-increasing rate.

And firms are realizing that it’s possible for them to use business analytics to gain the same types of actionable insight from big data that analytic rock stars like Google, Amazon and Target have been gathering for years.

question mark 1070028cl 8 150x150 Data Discovery: 6 Questions to Ask for Real Insight From Big Data But how does a firm figure out how to gain market advantage by tapping the real value of big data? And, at the same time, how does it avoid getting snagged by all the “noise,” the irrelevant data that flows in with the good stuff?

“Progressive firms use big data to narrow their decision set, not expect an answer,” notes Paul Magone, consultant and Forbes contributor. “They have figure[d] out how to map the real-time information against historical information to offer predictive possibilities. The modern crystal ball is thus unveiled and companies await the next prediction.”

He goes on to note six questions that companies need to ask to get real business insight from big data.

1. Determine the real business question: What issue will you clarify to truly move your business forward? Companies must ensure that they understand and focus on the business impact of analytical outcomes, according to research from Gartner Inc. The focus on material and measurable decisions is critical.

Decisions must mean something significant to the business, and the net benefits of the investments must be measured so that there is a clear relationship between the data, analysis and resulting decisions and outcomes. Ask business leadership to define the most valuable processes, and focus analytics on improving decisions around those processes, Gartner advises.

2. Understand that you or your organization may have bias toward the data: As an individual, will you take an analytic or creative view of the results? Will you and the stakeholders use the results discovered to inform or compel action in the organization? A major barrier to success in exploiting big data with analytics to bolster company performance is that the long-standing existence of silos of information across organizations has created an inherent lack of confidence in the accuracy and usefulness of data, Gartner notes in separate research.

To overcome these issues, Gartner recommends that firms create stakeholder analysis to identify cultural roadblocks to data sharing and prepare communications to overcome these obstacles. Companies like Proctor & Gamble, for example, have found success at advocating open innovation efforts that allow customers to participate directly in product development, which requires and inspires cross-team data sharing.

3. Determine the best data to answer the essential question you are probing: Is it data at rest, data in motion or data in use? Is the data trustworthy? Is the data volatile and incomplete? Finding the best data to answer the essential business questions first requires finding the right tools and methods to analyze the information quickly enough to impact business decisions. In a recent survey, Aberdeen finds that  28% of organizations need to provide information about business events to managers within one hour of when they occur for them to make timely management decisions.

In a separate, unpublished survey, Aberdeen finds that 64% of business managers have seen their decision windows shrink in the last 12 months. Aberdeen notes that 44% of the organizations that use visual and interactive analytics tools are always able to provide business managers with the information they need within the timeframes they require. Only 17% of the companies that depend on traditional BI tools are able to consistently get information into the hands of decision makers in the time required.

4. What are the best methods available for you to collect the data? Be sure to factor in social media tools, mobility and localization. Rather than trying to collect everything, seek just enough to get clear answers.

5. What’s the best way for you to harvest the insights? Beyond analysis this requires impartial investigation that goes beyond surface level inspection. Ensure the data is put in context against business issues that matter.

6. How can you make the best recommendations for the business? Determine the recommendations based on the data discovery and overall needs of the stakeholders. Take note if the recommendations mesh with your long-term strategy or short-term tactics.

Next Steps: See how Spotfire version 4.5 empowers users to discover actionable insights hidden in big data and unstructured information in our on-demand webcast, “What’s New with Spotfire 4.5.”

Print post


Defining Business Analytics – Recap of the May 17 DM Radio Show Featuring Spotfire’s Mark Lorion

2012 the year analytics means business1 Defining Business Analytics   Recap of the May 17 DM Radio Show Featuring Spotfires Mark LorionBig data analytics. Predictive analytics. Reporting. Text mining. Business intelligence. These terms circulate around our industry and fall under a blanket category called business analytics.

But with all of these terms comes confusion and a vague understanding of what fits where.

Continue reading »

Print post


What’s Your Company’s Digital IQ?

collage digital iq2 150x150 Whats Your Company’s Digital IQ?Many companies are accustomed to the concepts of market intelligence, competitor intelligence and customer intelligence.

But with the explosion in the amount of data available to companies from internal sources and scattered across the Web, creating actionable analysis to solve critical business problems requires a new type of intelligence – digital intelligence.

Continue reading »

Print post


Using Cell Phone Data for Social Good

data from cellphone companies 150x150 Using Cell Phone Data for Social GoodWhen is cell phone data not just cell phone data? When it’s being mined to solve some of the world’s biggest social problems – that’s when.

Which is exactly what Nathan Eagle is doing.

Eagle, a professor at Harvard’s School of Public Health and the MIT Media Lab, and his team are collecting and analyzing millions of phone records generated each day by mobile phone subscribers around the world. Although the data is typically collected for billing purposes, it can also be used to do a lot of social good, according to the article.

Continue reading »

Print post