Maximum data records Spotfire can import

TIBCO Spotfire Community » TIBCO Spotfire » TIBCO Spotfire Clients » Maximum data records Spotfire can import

Last post Tue, Aug 31 2010 1:01 AM by prathibha. 3 replies.

Page 1 of 1 (4 items)
	Sort Posts: Previous Next

Mon, Aug 30 2010 6:27 AM

prathibha
Joined on Mon, Aug 9 2010
Posts 5

Maximum data records Spotfire can import

Reply Contact

Hi All,

I tried importing 12 million records from oracle database to spotfire client 3.1.

Spotfire stopped responding.

Is there a data limitation to import data into Spotfire 3.1 ?

If not what might be the reason for Spotfire not responding?

Thanks in Advance

Prathibha

Mon, Aug 30 2010 7:07 AM In reply to

EranS
Joined on Mon, Oct 12 2009
Israel
Posts 376

Re: Maximum data records Spotfire can import

Reply Contact

Hi Prathibha,

I dont know of any limitation other then the size of your RAM and Operating system.

I have worked with more then 160M records from mysql DB.

When we tried with win-xp 32bit . The outcome was similair to yours. Win XP is limited to 1.5GB of RAM no matter how much you really have.

After moving to win2003 64bit there were no more limits.

For more info on recommended hardware and O/S please refer to Tibco Spotfire support or official system requirements datasheets.

Best Regards,

Eran

Mon, Aug 30 2010 7:41 AM In reply to

Christof
Joined on Mon, Feb 18 2008
Germany
Posts 1,186

Re: Maximum data records Spotfire can import

Reply Contact

Hi,

when you use Spotfire on a 64-bit Windows 7 System ther is no limit. I have tested with huge datasets on a mchine with 24GB RAM and it works great.

When Spotfire stops responding while loading data most probably your RAM is full and it moves data out to disk (we use our own swap mechanism.

It will still load the data, but it will be slow due to frequent disk access compared to in-memory only.

We have a lot of functionality built around the issue of loading huge datasets. E.g. you can use an aggregated data table (small) as starting point and drill down into the row-level data using an "on-demand" Information Link.

By this you can load only the portion of data you are interested in, and you don't have to wait for the big data table to load.

Kind regards,

Christof

-----------------------------------
Dr. Christof Gaenzler
Sr Solutions Consultant
TIBCO Spotfire

Filed under: 64-bit, swap

Tue, Aug 31 2010 1:01 AM In reply to

prathibha
Joined on Mon, Aug 9 2010
Posts 5

Re: Maximum data records Spotfire can import

Reply Contact

Hi Eran/Christof:

Thanks for the information. This clears the doubt we had for sometime now.

Will experiment and I am positive of the results.

Cheers!

Prathibha

Page 1 of 1 (4 items)

Spotfire's interactive information visualization and analytic solutions give users a remarkable experience for quickly and easily querying data and reporting results for superior business intelligence. From portfolio management and customer retention programs to key processes such as CRM, marketing, research, bioinformatics, yield and asset management and design for manufacturing, enterprises around the world rely on Spotfire's business analytics software to improve operational performance.