Trends and Outliers
TIBCO Spotfire's Business Intelligence Blog
Category Archives: Data Mining
2011
Top Three Challenges for Data Miners
An article in the June issue of Data Analytics magazine discusses the results of the 2010 Rexer Analytics annual data mining survey. Different from surveys that target analytics executives, the Rexer survey goes direct to the data miners themselves. In the 2010 survey, data miners identified their top challenges and many discussed how they’ve tried to work around or overcome obstacles. In both cases, a clear theme emerges that has very little to do with statistics and a lot to do with engaging and communicating with business users.
1. Dirty Data
It’s no surprise to Rexer that dirty data tops the list, because it has been at the top of the list for the past several years. It’s probably no big surprise to anyone who reads our blog either, because we’ve discussed in several posts how dirty data can derail data analytics and business intelligence projects. In the Rexer survey, many data miners provided input as to how they’ve tried to overcome the problem, and a clear theme emerges: involve business users. Data miners use descriptive statistics and visualization to assist business users in understanding their data and identifying problem areas. Helping users understand their data “hands on” helps everyone gain a shared understanding of the quality of the data. This can help manage expectations about the potential results of a data modeling exercise given data quality and convince data owners to create action plans to improve quality.
2011
“Data Geeks” in Demand as Salaries Climb in 2011
OK, so your company has taken a big step toward operating more efficiently by implementing a business intelligence (BI) solution—or it’s planning to install one in the near future. But if that’s the case then you’re probably also going to have to enlist the help of a new type of worker—a “data geek”—to make sense of all that BI data.
At least that’s the opinion of Jorge Garcia, a research analyst at Technology Evaluation Centers Inc. Organizations that use business analytics software can improve their decision-making processes if they have the right set of tools and the right people—those data geeks—in place, Garcia says.
2011
New Degree Programs Help Meet the Demand for Analytic Talent
Continue reading »
According to the U.S. Bureau of Labor Statistics, the demand for people with analytics skills and expertise is growing much faster than other occupations. BLS forecasts more than 20 percent growth in demand for mathematicians, operations research analysts and management analysts across all industries. To help provide the educational background needed for many of these roles, colleges and universities are introducing new degree and certificate programs focused on data mining, data analytics and business intelligence. Below are a few examples:
2011
Data Mining and Predictive Analytics Contest Has a $3 Million Prize
Continue reading »
In a post earlier this year entitled, “Data Mining Improved Company’s Revenue By 187%,” we described how the knowledge that led to a 187% improvement in revenue was hiding inside data that Assurant Solutions collected, but did not use. A California managed care provider also believes there is hidden knowledge inside data that they have, but aren’t using. The Heritage Provider Network hopes to discover knowledge in patient healthcare data that can prevent hospital stays and reduce healthcare expenses. To find that knowledge, they’re turning to the public for help and offering a $3 Million prize to the team who can come up with the best and most accurate solution.
Filed under: Data Mining, Predictive Analytics
2011
Data Mining Improved Company’s Revenue By 187%
Continue reading »
Assurant Solutions sells insurance policies on credit card payments, for example if you lose your job or become disabled. Their customer service center takes calls from people who see the monthly add-on fee on their bill and call to cancel. Metrics are simple. If a customer does not cancel, the customer is “saved.” If a customer is saved by downselling or upselling to a different plan, a percentage of revenue is saved. Assurant’s objective was to increase the percentage of saved customers and to increase the percentage of saved revenue. To do this, they agreed to abandon their assumptions about how the customer service center should operate in favor of what data mining revealed. MIT Sloan Management Review published this article detailing their astounding results.
2011
The ABCs of Desktop Data Mining
What is it?
“Data mining” can be described as the process of finding patterns and relationships in data. There are several different flavors of data mining, using different methods and pursuing different results. For example, a data mining project might look for:
- associations (interconnected events or information items)
- sequential relationships (one event leading to another)
- affinities (events that occur in clusters, information items that are frequently found together)
There are also two levels of approach to data mining. “True” data mining is an IT-intensive process, using complex algorithms and very sophisticated procedures to discover deep and/or unexpected patterns in data. At the business level, however, data mining is often viewed more generally, as a way of exploring data to answer questions and support analysis.
Continue reading »
Filed under: Business Intelligence Guide, Data Mining
Recent Comments