Add to Technorati Favorites

 Subscribe in a reader

Trends and Outliers

TIBCO Spotfire's Business Intelligence Blog

Category Archives: Data Mining


Big Data, Financial Services and Consumer Credit

Big data, once relegated to the technology industry, is making its way toward financial institutions looking to make better decisions about consumer loans and credit opportunities.

shutterstock 158046419 300x200 Big Data, Financial Services and Consumer CreditThe end result? Helping financial institutions around the world gather more data, analyze it more effectively, and make decisions more quickly.

Big Data, Big Changes

It’s been business as usual in the banking industry for almost 40 years.

The fundamental process of banking has remained the same despite rapid shifts in consumer technology because banks have continued to use the same formula to make decisions: gather as much data as possible, and ask a series of logical questions about it.

Continue reading »

Print post


Data Analysis to Build Consumer Trust

Like Verizon and Facebook, AT&T is also planning to sell customers’ smartphone data to third parties for marketing purposes.

trust2 Data Analysis to Build Consumer TrustUnder its forthcoming privacy policy, AT&T has announced that it will deliver more relevant advertising to customers based on the apps that they use along with their locations, made possible through GPS tracking capabilities.

The company has also offered assurances to its customers that any data that’s shared with third parties would be made anonymous to protect individual identities.

Continue reading »

Print post


Data Analysis: Tracking Tornadoes, Terrorists and Bombs

Increasingly, federal agencies are tapping the power of big data and analytics to better perform a myriad of tasks, including predicting killer tornadoes, thwarting terrorist operations and disrupting the networks in Iraq and Afghanistan that build improvised explosive devices (IEDs).

weatherupdate Data Analysis: Tracking Tornadoes, Terrorists and BombsAs most of the East Coast hunkers down and braces for Hurricane Sandy, what better time to tell you how federal data scientists are using data analysis to track weather events, among other things.

For example, the National Weather Service is combining big data streaming in from its satellites with automated weather services to create models that are aimed at identifying potential tornadoes quickly enough so that the public can be warned.

Continue reading »

Print post


3 Questions to Ask to Jumpstart Your Data Analytics Team

Tampa International Airport is playing its own version of Moneyball – and winning – as it has turned to data analytics to help it compete more effectively against larger airports in Miami and Orlando.

The airport uses analytics and data mining to uncover evidence that a particular airline could make money by offering flights to and from the Tampa Airport. Then it takes that data to the airline to convince it to add flights.

jump 1717 3 Questions to Ask to Jumpstart Your Data Analytics TeamThe effort has paid off as data analytics has helped the airport secure new daily routes to Cuba and Switzerland this year, boosting international travel by 20% in the first seven months of 2012.

Continue reading »

Print post


Using Data Mining to Pinpoint What Customers Really Want

One of the greatest strengths of data mining is its ability to provide companies with revealing and compelling insights into the needs and preferences of their customers.

This includes information that customers share about themselves in social media channels, in contact center interactions, and through their online behaviors.

target audience 150x150 Using Data Mining to Pinpoint What Customers Really WantThrough the sentiments they share and the actions they take, customers convey the types of products they’re interested in as well as the services, processes, and policies used by companies that delight or infuriate them.

As Liana Evans says in recent blog, customers reveal a great deal about their interests and their likes and dislikes through comments shared each day on Facebook alone. “When we hear one suggestion over and over again, we know we have to take the next step . . . ,” adds Evans.

Of course, there are numerous ways to identify and act on the information that customers are sharing across multiple channels beyond Facebook. For instance, web mining allows retailers and other types of companies to detect patterns in online customer behaviors. As wiseGEEK notes, structure mining examines how customers are using web sites, including the types of pages they’re visiting and the information being sought.

Continue reading »

Print post


After all the Hoopla: Three Real Big Data Apps

hype free zone 150x150 After all the Hoopla: Three Real Big Data AppsThe cacophony of hype surrounding big data is becoming deafening.

As with all technology that gets caught in top billing as the next big thing, mining big data to unearth its secrets won’t be the silver bullet for every company for every problem today or even tomorrow.

But when big data “grows up,” it definitely will have a major impact in many areas, specifically enterprise BI, government applications and customer relationship optimization, according to Alistair Croll, a founding partner at startup accelerator Year One Labs and an analyst at Bitcurrent.

For decades, analysts have been using business intelligence tools to crunch large amounts of data and answer straightforward questions like: “What are each sales representative’s sales for the month?” But these tools have struggled with predictive analytics that ask the “what if” questions that can help guide company strategy, Croll says.

Continue reading »

Print post


Using Big Data to Make Big Money

dollar sign finance 150x150 Using Big Data to Make Big MoneyOne of the greatest strengths of big data is how it can be used to provide fresh insights to decision makers.

Big data can reveal customer and market trends that, when spotted quickly enough, can lead executives to move rapidly on new business ventures ahead of competitors.

The beauty of the increasingly digital landscape is that every single digital interaction that occurs – from scheduling a doctor’s visit via email to entering a chat session with a customer agent regarding a product issue to commenting about a brand experience on Facebook – becomes an electronic record that companies can make use of, notes Inc. magazine reporter J.J. McCorvey in a recent blog.

And businesses can now take advantage of this “treasure trove” of data thanks, in part, to dramatic reductions in the costs of storage, the ability to integrate data streams from multiple sources, and the rich analytics tools that are available to mine and analyze this information,” says Todd Nash of Chicago Business Intelligence Group.

Continue reading »

Print post


Big Data Mining Shift: From Mining Transactions to Reverse-Engineering the Brain

reverse engineer brain 150x150 Big Data Mining Shift: From Mining Transactions to Reverse Engineering the BrainThe paradigm shift from businesses struggling to pinpoint what data to store to what they can do with the infusion of big data is in full swing.

Transactional data is still the foundation for many businesses trying to mine data for insights, but big data has opened an entirely new realm of data mining prospects for a multitude of industries.

Instead of simply modeling data, big data provides the opportunity to model human intent, notes Mok Oh, chief scientist of PayPal.

“Ultimately, what we’re trying to model is every person’s brain – at least the part of the brain that decides how to shop, when to shop, and what you want,” Oh says. “We’re trying to reverse-engineer transactional data to figure out what people are going to buy next.”

Continue reading »

Print post


Using Cell Phone Data for Social Good

data from cellphone companies 150x150 Using Cell Phone Data for Social GoodWhen is cell phone data not just cell phone data? When it’s being mined to solve some of the world’s biggest social problems – that’s when.

Which is exactly what Nathan Eagle is doing.

Eagle, a professor at Harvard’s School of Public Health and the MIT Media Lab, and his team are collecting and analyzing millions of phone records generated each day by mobile phone subscribers around the world. Although the data is typically collected for billing purposes, it can also be used to do a lot of social good, according to the article.

Continue reading »

Print post


How To Develop A Top-Notch Data Warehousing System

DataExplosion How To Develop A Top Notch Data Warehousing SystemUnlike just a few years ago, each of us generates a “ton” of data every day.

Almost everything we do generates date including surfing the web; buying groceries from the supermarket; and sending text messages. In fact, with the right mobile technology, simply walking into our local malls generates data.

With this explosion of data, developing a top-notch data warehousing system is paramount to the success of any company. Processing data in a well-developed data warehousing system provides the competitive edge that companies are striving for. The question then becomes: What steps should you take to ensure you build a data warehouse that enhances and supports the decision-making process?

Mike Ferguson makes the point that you and your data warehouse development team have to understand certain things including:

  • The corporate business strategy: For your decision makers to make the best use of the data, your data warehouse development team must understand the corporate business strategy. Once they do, they can work with the decision makers to determine which objectives are priorities. They can also figure out how the data can lead to a higher return on investment. They must continue to work with the decision makers to join the data elements to the objectives then determine how to capture the appropriate data and build the necessary dependencies to make the data meaningful.
  • The data requirements: Your development team must work with the corporate data scientists and analysts to define data requirements, data sets and desired data visualizations to ensure that the data warehouse is highly interactive and that it allows users to customize the data sets, charts, and graphs. And they must make that information available in a variety of formats including dashboards, scorecards, and reports.
  • The technical environment: You must learn as much as you possibly can about the proposed technology and use that knowledge to draft the data warehouse technical architecture. Then you should participate in all facets of the technology selection process and work with the team to develop an implementation plan, which should include:
    • A metadata repository – to track information about both the data and the system including processes. Define the business vocabulary, store it within the repository and share it across the organization.
    • An ETL process – to extract data from transaction systems, transform the data into something suitable for the data warehouse and then load the data into the target system; i.e. the data warehouse. Pay close attention to how the data is manipulated, how long the process takes to completely execute, and the accuracy of the data. Be prepared for data that may be incomplete, incomprehensible, and inconsistent and develop processes to handle the issues.
    • Security – what level of security will be required? How will the data warehousing system be maintained to the level required by the organization, internal compliance, and external laws and regulations?
    • Data integration – combine data from several disparate sources into a unified data warehousing system. Then attempt to embed the system within existing corporate software for quicker adoption as the user may feel familiarity with the look and feel, leading to less end-user training.

Next Steps: 

  • To learn more about how analytics can improve your business and increase your bottom line check out these complimentary guides:
    • 5-Minute Guide to Business Analytics,” to find out how user-driven “analytic” or “data discovery” technologies help business and technology users more quickly uncover insights and speed action.
    • 5-Minute Guide to HR Analytics,” to discover the three critical capabilities a modern analytic environment must provide to the entire spectrum of HR staff so they can adequately support the enterprise.
    • 5-Minute Guide to CRM Analytics,” to learn how agile analytics technology can help you deliver critical value to executives and front-line marketers.

Dennis Hardy
Spotfire Blogging Team

Print post