Knowledge Base

Loading images from an URL using Data Functions

2010-03-15T09:42:00Z

In this example we will show how to write a simple data function executor extension that load images from a column of URLs and returns the images as binary large objects.

First we need to create a function executor which takes in an input parameter called “url” which contains strings that are supposed to be URLs and returns an output parameter names “image” that is a list of images as binary large objects.

namespace ImageFromURL
{
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Net;
    using System.Security;
    using Spotfire.Dxp.Application.Extension;
    using Spotfire.Dxp.Data;
    using Spotfire.Dxp.Data.Collections;

    /// <summary>
    /// This is a simple function executor that loads images from URLs.
    /// </summary>
    public class ImageFromURLFunctionExecutor : CustomDataFunctionExecutor
    {
        /// <summary>
        /// Execute a function invocation. The executor is expected to add the output results to the invocation
        /// object.
        /// </summary>
        /// <param name="invocation">The function invocation.</param>
        /// <returns>
        /// The prompt models to return, the implementor is expected to use the yield return pattern.
        /// </returns>
        protected override IEnumerable<object> ExecuteFunctionCore(Spotfire.Dxp.Data.DataFunctions.DataFunctionInvocation invocation)
        {
            // Get the input as a row reader, this one always expect an url parameter.
            DataRowReader urlReader = invocation.GetInput("url");

            // Create a pageable list to store the result in.
            PageableListSettings settings = new PageableListSettings();

            // Since we only write the data sequentially we can get better performance by not
            // allowing random writes.
            settings.CanReplaceValue = false;
            PageableList<BinaryLargeObject> output = new PageableList<BinaryLargeObject>(settings);

            // Cursor that contains the input URL
            DataValueCursor<string> cursor = (DataValueCursor<string>)urlReader.Columns[0].Cursor;            

            // Loop over all input rows.
            while (urlReader.MoveNext())
            {
                if (!cursor.IsCurrentValueValid)
                {
                    // Invalid value as input so return invalid value as output.
                    output.Add(null);
                    continue;
                }

                string url = cursor.CurrentValue;

                try
                {
                    // Use the standard web request API to retrieve the images.
                    Uri uri = new Uri(url);

                    WebRequest webRequest = WebRequest.Create(uri);

                    WebResponse response = webRequest.GetResponse();

                    MemoryStream memoryStream = new MemoryStream();
                    Stream responseStream = response.GetResponseStream();
                    byte[] buf = new byte[8 * 1024];
                    int n = 1;
                    while (n > 0)
                    {
                        n = responseStream.Read(buf, 0, buf.Length);
                        memoryStream.Write(buf, 0, n);
                    }

                    memoryStream.Seek(0, SeekOrigin.Begin);

                    BinaryLargeObject image = BinaryLargeObject.Create(memoryStream);
                    output.Add(image);
                }
                catch (UriFormatException)
                {
                    output.Add(null);
                }
                catch (NotSupportedException)
                {
                    output.Add(null);
                }
                catch (SecurityException)
                {
                    output.Add(null);
                }
            }

            // Assign the result.
            invocation.SetOutput("image", new ImageReader(output));

            // No need to prompt so just break.
            yield break;
        }

        /// <summary>
        /// Simple reader to return the images.
        /// </summary>
        private class ImageReader : CustomDataRowReader
        {
            /// <summary>
            /// The list of the images as binary large objects.
            /// </summary>
            private readonly PageableList<BinaryLargeObject> list;

            /// <summary>
            /// The cursor to assign the result for the current row.
            /// </summary>
            private readonly MutableValueCursor<BinaryLargeObject> cursor;

            /// <summary>
            /// The columns returned by the reader.
            /// </summary>
            private readonly List<DataRowReaderColumn> columns;

            /// <summary>
            /// The current index to return.
            /// </summary>
            private int index = -1;

            /// <summary>
            /// Initializes a new instance of the <see cref="ImageReader"/> class.
            /// </summary>
            /// <param name="list">The list of the images as binary large objects.</param>
            public ImageReader(PageableList<BinaryLargeObject> list)
            {
                this.list = list;
                this.columns = new List<DataRowReaderColumn>();
                this.cursor = (MutableValueCursor<BinaryLargeObject>)DataValueCursor.CreateMutableCursor(DataType.Binary);

                // Say that the result is of type image/png..
                DataColumnProperties properties = new DataColumnProperties();
                properties.ContentType = "image/png";

                // Create the metadata for the put column.
                DataRowReaderColumn column =
                    new DataRowReaderColumn(
                        "Image",
                        DataType.Binary,
                        properties,
                        this.cursor);

                this.columns.Add(column);
            }


            /// <summary>
            /// The implementor should provide the result properties.
            /// </summary>
            /// <returns>The result properties.</returns>
            /// <remarks>
            /// This method is only called once.
            /// </remarks>
            protected override ResultProperties GetResultPropertiesCore()
            {
                // Table metadata can be returned here, but we have nothing to return for now.
                return new ResultProperties();
            }

            /// <summary>
            /// The implementor should implement this method to reset the
            /// enumerator. If this method is called then the <see cref="M:Spotfire.Dxp.Data.DataRowReader.MoveNextCore"/>
            /// method should return the first row again when called the next time.
            /// </summary>
            protected override void ResetCore()
            {
                this.index = -1;
            }

            /// <summary>
            /// The implementor should provide a list of <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s that it
            /// returns.
            /// </summary>
            /// <returns>The metadata of the columns that are returned by the reader.</returns>
            /// <remarks>
            /// This method is only called once.
            /// </remarks>
            protected override IEnumerable<DataRowReaderColumn> GetColumnsCore()
            {
                return this.columns;
            }

            /// <summary>
            /// Advance to the next row.
            /// The implementor should update all <see cref="T:Spotfire.Dxp.Data.DataValueCursor"/>s in the <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s
            /// with values for the next row.
            /// </summary>
            /// <returns>
            ///    <c>true</c> if there are more rows; otherwise <c>false</c>.
            /// </returns>
            protected override bool MoveNextCore()
            {
                this.index++;
                if (this.index >= this.list.Count)
                {
                    // We are done, past the final row.
                    return false;
                }

                // Get the image for the current row and assign to the cursor.
                BinaryLargeObject image = this.list[this.index];
                if (image == null)
                {
                    this.cursor.MutableDataValue.SetNullValue();
                }
                else
                {
                    this.cursor.MutableDataValue.ValidValue = image;
                }

                return true;
            }
        }
    }
}

We also need to create a type identifier for the executor which is needed for registration.

namespace ImageFromURL
{
    using Spotfire.Dxp.Application.Extension;

    /// <summary>
    /// Type identifiers for the image function exectuor.
    /// </summary>
    public class TypeIdentifiers : CustomTypeIdentifiers
    {
        /// <summary>
        /// The image from URL executor.
        /// </summary>
        public static CustomTypeIdentifier ImageFromURLExecutor = CreateTypeIdentifier("ImageFromURL", "Image from URL.", "Downloads an image from an URL");
    }
}

Now we are almost done, but we need to have a DataFunctionDefinition for this executor. To make this simple we create a tool that creates a function definition and stores it in the library so that we can use the default DataFunctions UI to execute it. This tool creates and saves the function definition in the library, it only needs to be run once then the function definition could just be import and exported to be moved to a different Spotfire server. Since we only need to run this one we make it very simple and assumes that there is a folder called “imagefromurl” in the library where we store the definition. We can always move it later once it has been stored.

namespace ImageFromURL
{
    using Spotfire.Dxp.Application;
    using Spotfire.Dxp.Application.Extension;
    using Spotfire.Dxp.Data;
    using Spotfire.Dxp.Data.DataFunctions;
    using Spotfire.Dxp.Framework.Library;

    /// <summary>
    /// One-off tool to create and save a function definition in the library.
    /// Assumes that there is an imagefromurl folder in the library.
    /// </summary>
    public class RegisterFunctionTool : CustomTool<AnalysisApplication>
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="RegisterFunctionTool"/> class.
        /// </summary>
        public RegisterFunctionTool()
            : base("Register Image from URL tool.")
        {               
        }

        /// <summary>
        /// Implement this method to perform the tool-specific logic.
        /// </summary>
        /// <param name="context">The context of the tool. Is never <c>null</c>.</param>
        /// <remarks>Note that this method is only called if
        /// <paramref name="context"/> is not <c>null</c> and
        /// <see cref="M:Spotfire.Dxp.Application.Tool`1.IsEnabled(`0)"/>
        /// returns <c>true</c>.</remarks>
        protected override void ExecuteCore(AnalysisApplication context)
        {
            // Create the function definition builder.
            DataFunctionDefinitionBuilder builder =
                new DataFunctionDefinitionBuilder("Image from URL", TypeIdentifiers.ImageFromURLExecutor);

            // One input argument called url which is a string column.
            InputParameterBuilder inputBuilder = new InputParameterBuilder("url", ParameterType.Column);
            inputBuilder.AddAllowedDataType(DataType.String);
            builder.InputParameters.Add(inputBuilder.Build());

            // One output parameter called image which is a column.
            OutputParameterBuilder outputBuilder = new OutputParameterBuilder("image", ParameterType.Column);
            builder.OutputParameters.Add(outputBuilder.Build());

            DataFunctionDefinition functionDefinition = builder.Build();

            LibraryManager lm = context.GetService<LibraryManager>();

            // Find the folder where it should be stored.
            LibraryItem li = lm.Search("imagefromurl")[0];

            // Save the function definition in the library.
            DataFunctionDefinition saved;
            functionDefinition.SaveAs(li, functionDefinition.FunctionName, new string[] { }, out saved);
        }
    }
}

To finish things, we finally add the AddIn to register the tool and function executor.

namespace ImageFromURL
{
using Spotfire.Dxp.Application.Extension;

    /// <summary>Addin registration.
    /// </summary>
    public sealed class CustomAddIn : AddIn
    {
        /// <summary>
        /// Override this method to register data function executors with the application.
        /// </summary>
        /// <param name="registrar">Object responsible for registering new data function executors.</param>
        protected override void RegisterDataFunctionExecutors(AddIn.DataFunctionExecutorRegistrar registrar)
        {
            base.RegisterDataFunctionExecutors(registrar);

registrar.Register(TypeIdentifiers.ImageFromURLExecutor, new ImageFromURLFunctionExecutor());
}

        /// <summary>
        /// Override this method to extend the application with new tools.
        /// Each new tool should inherit from <see cref="T:Spotfire.Dxp.Application.Extension.CustomTool`1"/>,
        /// and is then registered with the <see cref="T:Spotfire.Dxp.Application.Extension.AddIn.ToolRegistrar"/>.
        /// </summary>
        /// <param name="registrar">Object responsible for registering new tools.</param>
        protected override void RegisterTools(AddIn.ToolRegistrar registrar)
        {
            base.RegisterTools(registrar);

            registrar.Register(new RegisterFunctionTool());
        }
    }
}

Now we can first run the “Register Image from URL Tool” from the Tools menu and then use the Tools->Data Functions menu item to run the image from URL function.

We then select to add columns with the result:

Finally we press OK and the images are retrieved and added to the table.

Conditional Coloring to Highlight Top and Bottom Values

2010-03-13T19:15:00Z

In this week's post, we will introduce you to another excited new feature of TIBCO Spotfire 3.1, Conditional Coloring. Let’s assume you have a Data Table which provides sales data for a particular store. The sales are broken up into three separate product categories: Furniture, Office Supplies, and Technology, and sales are measured across the different store locations per state. Let’s assume we want to highlight the top 5 sales result and the bottom 5 sales result for the combination of products...(read more)

Introduction to Data Functions

2010-03-12T14:44:00Z

One of the major new APIs in TIBCO Spotfire 3.1 is data functions. The data functions API is used to connect to TIBCO Spotfire Statistics Services but it can be used for much more than that. Operations that was previously performed using the data source, transformations and calculations API can now also be performed in the new API. The advantage of the data functions API is that it automatically takes care of executing operations on background threads to easily take advantage of multi-core CPUs.

Conceptually the data functions API is split into three different parts:

The DataFunctionDefinition which describes the metadata about what a data function expects for input and the output provided by the function. These definitions can be stored and retrieved from the library and are not connected directly to any particular document.
The DataFunction which includes a function definition and also specifies how input and outputs should be retrieved from the document.
The DataFunctionExecutor which receives input from the document combined with the function definition and is responsible for calculating the output.

Have a look at the DataFunctionsExample project in the SDK for code examples on how to use the API.

Compared to the threading framework

This component uses the threading framework but does not replace it. However for many operations such as calculations and data modification operations that one wants to execute in a background thread this is the preferred way to do it. Calculations written using data functions will automatically be executed on a background thread while still you only need to write simple sequential code.

Compared to calculations

In many cases it is preferable to use data functions rather than the calculations framework since you will get automatic background execution and the data functions API should be easier to use. However the calculations are more general and are needed for some scenarios.

Compared to transformations

Data functions are quite similar to transformations since they both takes have an API that is based on the DataRowReader enumerators, however data functions are more general since it can take multiple row readers as input and return multiple row readers as output. Simple data functions can automatically be used as transformations by using the DataFunctionTransformation. It should also be easy to expose an existing transformation by wrapping it in a data function executor.

Compared to data sources

Since data sources returns data row readers and an ImportContext is available to the executor in the DataFunctionInvocation object it is be possible to wrap a data source inside a data function executor.

DataFunctionDefinition

The data function definition object and child objects is immutable and a build class DataFunctionDefinitionBuilder is used to construct instances. To support the use-case where you want to edit a definition it is possible to create a build from a definition in which case the builder will be populated with the previous definition input.

Each function definition is connected to a TypeIdentifier which specifies how the function should be executed, the pre-defined type identifiers can be found in the DataFunctionExecutorTypeIdentifiers static class.

The design of the function definition is based on the assumption that these definitions should be applicable to large range of possible implementations such as web services, internal calculation functions or an external executable. In order to support this, a dictionary from string to string is included to allow custom information to be stored inside the definition. For S+ and R scripts the script is added to the settings with the key “script”.

Here is an example that shows how to create a function definition for a simple script that performs loess smoothing:

DataFunctionDefinitionBuilder functionDefinitionBuilder =
      new DataFunctionDefinitionBuilder(
         "Loess", 
         DataFunctionExecutorTypeIdentifiers.SPlusScriptExecutor);

InputParameterBuilder yBuilder =
      new InputParameterBuilder("y", ParameterType.Column);
yBuilder.AddAllowedDataType(DataType.Integer);
yBuilder.AddAllowedDataType(DataType.Real);

InputParameter y = yBuilder.Build();

functionDefinitionBuilder.InputParameters.Add(y);

InputParameterBuilder xBuilder =
      new InputParameterBuilder("x", ParameterType.Column);

InputParameter x = xBuilder.Build();

functionDefinitionBuilder.InputParameters.Add(x);

OutputParameterBuilder outputBuilder = new OutputParameterBuilder("smooth", ParameterType.Table);

OutputParameter output = outputBuilder.Build();

functionDefinitionBuilder.OutputParameters.Add(output);

DataTable inputTable = context.Data.DataTableReference;

StringBuilder scriptBuilder = new StringBuilder();

scriptBuilder.AppendLine("predicted <- predict(loess(y ~ x, data.frame(x=x, y=y)), data.frame(x=x))");
scriptBuilder.AppendLine("smooth <- data.frame(loess.x = x, loess.y = predicted)");

functionDefinitionBuilder.Settings.Add("script", scriptBuilder.ToString());

DataFunctionDefinition functionDefinition = functionDefinitionBuilder.Build();

DataFunction

The purpose of the data function is to allow functions to get input from the document, do some processing on a background thread and then apply the results to the document.

The inputs to a data function are expressions in the expression language, it multiple columns need to be sent then just separate the expression with commas as you would in the custom expressions dialog. It is possible to specify on which subset the expressions should be calculated on using any combination of markings and filtering. The Active Filtering property also allows you to make the data function use the filtering on the current page. If you specify multiple markings and filtering then the intersection will be used.

For the output the following operations are allowed:

Adding a new table.
Adding columns to a table, both as a join and directly added to the table.
Adding rows to a table.
Replacing all the data in a table.
Setting a document property.
Setting a column property.
Setting a table property.

The following example shows how to create a data function from the expressions in a scatter plot and also update the scatter plot when the calculation has finished:

Document document = context.Context.GetService<Document>();

DataSelection[] selections = new DataSelection[context.Data.Filterings.Count];
for (int i = 0; i < context.Data.Filterings.Count; ++i)
{
    selections[i] = context.Data.Filterings[i];
}

string xExpression = null;
string yExpression = null;
try
{
    ColumnExpression xColumnExpression = ColumnExpression.Create(context.XAxis.Expression);
    ColumnExpression yColumnExpression = ColumnExpression.Create(context.YAxis.Expression);

    if (!xColumnExpression.IsValid && !yColumnExpression.IsValid)
    {
        return;
    }

    xColumnExpression.QualifyNames(inputTable.Name);
    yColumnExpression.QualifyNames(inputTable.Name);

    xExpression = xColumnExpression.Expression;
    yExpression = yColumnExpression.Expression;
}
catch (ExpressionException)
{
    return;
}

DataFunction function = document.Data.DataFunctions.AddNew(context.Title, functionDefinition);

function.Inputs.SetInput(x, xExpression, true, selections);
function.Inputs.SetInput(y, yExpression, true, selections);

// Add columns to the input table.
ColumnsOutputBuilder addColumnsBuilder =
    new ColumnsOutputBuilder(inputTable);
addColumnsBuilder.InputToAdaptSelectionsFrom = x;

function.Outputs.SetColumnsOutput(output, addColumnsBuilder);

ApplicationThread applicationThread = context.Context.GetService<ApplicationThread>();

function.Execute(
    delegate
    {
        applicationThread.InvokeAsynchronously(
            delegate
            {
                context.Transactions.ExecuteTransaction(
                    delegate
                    {
                        DataColumn xColumn = inputTable.Columns["loess.x"];
                        DataColumn yColumn = inputTable.Columns["loess.y"];

                        ColumnValuesLine fittingModel =
                            context.FittingModels.AddColumnValuesLine(
                                inputTable,
                                xColumn,
                                yColumn);

                        fittingModel.SortColumnReference = xColumn;
                        fittingModel.Enabled = true;
                    });

                // Start updating automatically
                function.UpdateBehavior = DataFunctionUpdateBehavior.Automatic;
            });
    });

DataFunctionExecutor

The data functions executor is the only extension point in the data functions API and one needs to inherit from the CustomDataFunctionExecutor class. This class has a very simple API with only one method that receives a DataFunctionInvocation object which contains a the named input parameters and the function is expected to set the output parameters with the calculated result.

The following example shows a subset of simple function executor that performs a simple K-means clustering algorithm.

    public sealed class KMeansDataFunctionExecutor : CustomDataFunctionExecutor
    {
        /// <summary>
        /// Execute a function invocation. The executor is expected to add the output results to the invocation
        /// object.
        /// </summary>
        /// <param name="invocation">The function invocation.</param>
        /// <returns>
        /// The prompt models to return, the implementor is expected to use the yield return pattern.
        /// </returns>
        protected override IEnumerable<object> ExecuteFunctionCore(DataFunctionInvocation invocation)
        {
            // This function only expects function definitions with two parameters:
            //   Values -> Table of double values.
            //   NumberOfClusters -> Integer value.
            DataRowReader valuesReader;
            if (!invocation.TryGetInput("Values", out valuesReader))
            {
                throw new InvalidOperationException("Could not find values in input.");
            }

            int numberOfClusters = GetNumberOfClusters(invocation);

            if (numberOfClusters <= 0)
            {
                throw new InvalidOperationException("Too few number of clusters.");
            }

            string resultName = GetResultName(invocation);

            // Get cursors for the input as double values.
            List<DataValueCursor<double>> numericCursors = new List<DataValueCursor<double>>();
            foreach (DataRowReaderColumn column in valuesReader.Columns)
            {
                DataValueCursor<double> numericCursor = column.Cursor as DataValueCursor<double>;
                if (numericCursor == null)
                {
                    throw new InvalidOperationException("Non double cursor sent as input.");
                }

                numericCursors.Add(numericCursor);
            }

            // An array that contains the values for a row.
            double[] row = new double[numericCursors.Count];

            // We keep it simple and select the first numberOfClusters rows to be the initial
            // center for the clusters.
            Cluster[] clusters = new Cluster[numberOfClusters];
            valuesReader.Reset();
            for (int i = 0; i < numberOfClusters; ++i)
            {
                if (!valuesReader.MoveNext())
                {
                    throw new InvalidOperationException("Number of clusters is larger than the available rows.");
                }

                FillRow(numericCursors, row);

                // Initialize the row with the current center.
                clusters[i].Initialize(row);
            }

            PageableListSettings settings = new PageableListSettings();
            settings.CanReplaceValue = true;
            PageableList<int> rowClusterIndex = new PageableList<int>(settings);

            // Calculate the initial cluster for each point.
            valuesReader.Reset();
            while (valuesReader.MoveNext())
            {
                FillRow(numericCursors, row);

                // Calculate the distance from all clusters and find
                // the nearest cluster.
                int nearestCluster = FindNearestCluster(row, clusters);

                // Store the resulting index and add the row as a member of the nearest cluster.
                rowClusterIndex.Add(nearestCluster);
                clusters[nearestCluster].AddMember(row);
            }

            // Iterate until the maximum number of iterations is each or if no change
            // has been performed.
            int maxIterations = 100;
            int iterationCount = 0;
            do 
            {
                // Calculate the centroids for all clusters.
                for (int i = 0; i < numberOfClusters; ++i)
                {
                    clusters[i].CalculateCentroid();
                }

                // Calculate the new nearest cluster for each point.
                valuesReader.Reset();
                bool hasChanged = false;
                int rowIndex = 0;
                while (valuesReader.MoveNext())
                {
                    FillRow(numericCursors, row);

                    // Calculate the distance from all clusters and find
                    // the nearest cluster.
                    int nearestCluster = FindNearestCluster(row, clusters);

                    int oldClusterIndex = rowClusterIndex[rowIndex];

                    // This point has changed cluster.
                    if (nearestCluster != oldClusterIndex)
                    {
                        hasChanged = true;
                        rowClusterIndex[rowIndex] = nearestCluster;
                        clusters[oldClusterIndex].RemoveMember(row);
                        clusters[nearestCluster].AddMember(row);
                    }

                    rowIndex++;
                }

                if (!hasChanged)
                {
                    break;
                }

                iterationCount++;
            } 
            while (iterationCount < maxIterations);

            // Assign the output result.
            invocation.SetOutput("K-Means", new PageableListDataRowReader(rowClusterIndex, resultName));

            // Done
            yield break;
        }

        /// <summary>
        /// Find the cluster that is nearest the row.
        /// </summary>
        /// <param name="row">The row.</param>
        /// <param name="clusters">The clusters.</param>
        /// <returns>The index of the nearest cluster.</returns>
        private static int FindNearestCluster(double[] row, Cluster[] clusters)
        {
            double minDistance = double.MaxValue;
            int nearestCluster = -1;
            for (int i = 0; i < clusters.Length; ++i)
            {
                double distance = clusters[i].Distance(row);
                if (distance < minDistance)
                {
                    minDistance = distance;
                    nearestCluster = i;
                }
            }

            return nearestCluster;
        }

        /// <summary>
        /// Fill the row array with values for the current row.
        /// </summary>
        /// <param name="numericCursors">The numeric cursors.</param>
        /// <param name="row">The row array to full.</param>
        private static void FillRow(List<DataValueCursor<double>> numericCursors, double[] row)
        {
            for (int j = 0; j < numericCursors.Count; ++j)
            {
                if (!numericCursors[j].IsCurrentValueValid)
                {
                    throw new InvalidOperationException("Invalid values are not allowed.");
                }

                row[j] = numericCursors[j].CurrentValue;
            }
        }

        /// <summary>
        /// Get result name.
        /// </summary>
        /// <param name="invocation">The invocation.</param>
        /// <returns>Get the result name.</returns>
        private static string GetResultName(DataFunctionInvocation invocation)
        {
            DataRowReader resultNameReader;
            if (!invocation.TryGetInput("ResultName", out resultNameReader))
            {
                throw new InvalidOperationException("Could not find the result name.");
            }

            if (resultNameReader.Columns.Count != 1)
            {
                throw new InvalidOperationException("The result name reader contains more than one argument.");
            }

            DataRowReaderColumn resultNameColumn = resultNameReader.Columns[0];
            DataValueCursor<string> resultNameCursor = resultNameColumn.Cursor as DataValueCursor<string>;

            if (resultNameColumn == null)
            {
                throw new InvalidOperationException("The result name input was not an string.");
            }

            if (!resultNameReader.MoveNext())
            {
                throw new InvalidOperationException("No rows in result name input.");
            }

            if (!resultNameCursor.IsCurrentValueValid)
            {
                throw new InvalidOperationException("The result name input was not valid.");
            }

            return resultNameCursor.CurrentValue;
        }

        /// <summary>
        /// Get the number of clusters sent in the invocation.
        /// </summary>
        /// <param name="invocation">The invocation.</param>
        /// <returns>The number of clusters.</returns>
        private static int GetNumberOfClusters(DataFunctionInvocation invocation)
        {
            DataRowReader numberOfClustersReader;
            if (!invocation.TryGetInput("NumberOfClusters", out numberOfClustersReader))
            {
                throw new InvalidOperationException("Could not find number of clusters in input.");
            }

            if (numberOfClustersReader.Columns.Count != 1)
            {
                throw new InvalidOperationException("The number of clusters reader contains more than one argument.");
            }

            DataRowReaderColumn numClustersColumn = numberOfClustersReader.Columns[0];
            DataValueCursor<int> numClustersCursor = numClustersColumn.Cursor as DataValueCursor<int>;

            if (numClustersColumn == null)
            {
                throw new InvalidOperationException("The number of clusters input was not an integer.");
            }

            if (!numberOfClustersReader.MoveNext())
            {
                throw new InvalidOperationException("No rows in number of clusters input.");
            }

            if (!numClustersCursor.IsCurrentValueValid)
            {
                throw new InvalidOperationException("The number of clusters input was not valid.");
            }

            int numberOfClusters = numClustersCursor.CurrentValue;
            return numberOfClusters;
        }

        /// <summary>
        /// A cluster.
        /// </summary>
        private struct Cluster
        {
            /// <summary>
            /// The location of the clusters.
            /// </summary>
            private double[] location;

            /// <summary>
            /// The sum of the parts of the points in the clusters.
            /// </summary>
            private double[] sumOfParts;

            /// <summary>
            /// The number of observations.
            /// </summary>
            private int observations;

            /// <summary>
            /// Initialize the cluster with the specified center.
            /// </summary>
            /// <param name="center">The row with the initial center.</param>
            public void Initialize(double[] center)
            {
                this.location = new double[center.Length];
                this.sumOfParts = new double[center.Length];
                this.observations = 0;

                for (int i = 0; i < center.Length; ++i)
                {
                    this.location[i] = center[i];
                }
            }

            /// <summary>Add a member to the cluster.
            /// </summary>
            /// <param name="member">The member to add.</param>
            public void AddMember(double[] member)
            {
                for (int i = 0; i < member.Length; ++i)
                {
                    this.sumOfParts[i] += member[i];
                }

                this.observations++;
            }

            /// <summary>Remove a member from the cluster.
            /// </summary>
            /// <param name="member">The member to remove.</param>
            public void RemoveMember(double[] member)
            {
                for (int i = 0; i < member.Length; ++i)
                {
                    this.sumOfParts[i] -= member[i];
                }

                this.observations--;
            }

            /// <summary>
            /// Calculate the new centroid.
            /// </summary>
            public void CalculateCentroid()
            {
                for (int i = 0; i < this.sumOfParts.Length; ++i)
                {
                    this.location[i] = this.sumOfParts[i] / this.observations;
                }
            }

            /// <summary>
            /// Calculates the euclidean distance.
            /// </summary>
            /// <param name="point">The point to calculate the distance to this cluster.</param>
            /// <returns>The distance from the cluster.</returns>
            public double Distance(double[] point)
            {
                double sum = 0;
                for (int i = 0; i < point.Length; ++i)
                {
                    double difference = this.location[i] - point[i];
                    sum += difference * difference;
                }

                return Math.Sqrt(sum);
            }
        }

        /// <summary>
        /// A data row reader which returns a pageable list.
        /// </summary>
        private sealed class PageableListDataRowReader : CustomDataRowReader
        {
            /// <summary>
            /// The data to return.
            /// </summary>
            private readonly PageableList<int> data;

            /// <summary>
            /// The result cursor.
            /// </summary>
            private readonly MutableValueCursor<int> resultCursor;

            /// <summary>
            /// The result name.
            /// </summary>
            private readonly string resultName;

            /// <summary>           
            /// The current row index.
            /// </summary>
            private int index = -1;

            /// <summary>
            /// Initializes a new instance of the PagableListDataRowReader class.
            /// </summary>
            /// <param name="data">The data to return.</param>
            /// <param name="resultName">The result name.</param>
            public PageableListDataRowReader(PageableList<int> data, string resultName)
            {
                this.data = data;
                this.resultName = resultName;
                this.resultCursor = (MutableValueCursor<int>)DataValueCursor.CreateMutableCursor(DataType.Integer);
            }

            /// <summary>
            /// The implementor should provide a list of <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s that it
            /// returns.
            /// </summary>
            /// <returns></returns>
            /// <remarks>
            /// This method is only called once.
            /// </remarks>
            protected override IEnumerable<DataRowReaderColumn> GetColumnsCore()
            {
                List<DataRowReaderColumn> columns = new List<DataRowReaderColumn>();

                DataRowReaderColumn column =
                    new DataRowReaderColumn(this.resultName, DataType.Integer, this.resultCursor);
                columns.Add(column);

                return columns;
            }

            /// <summary>
            /// The implementor should provide the result properties.
            /// </summary>
            /// <returns></returns>
            /// <remarks>
            /// This method is only called once.
            /// </remarks>
            protected override ResultProperties GetResultPropertiesCore()
            {
                return new ResultProperties();
            }

            /// <summary>
            /// Advance to the next row.
            /// The implementor should update all <see cref="T:Spotfire.Dxp.Data.DataValueCursor"/>s in the <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s
            /// with values for the next row.
            /// </summary>
            /// <returns>
            ///    <c>true</c> if there are more rows; otherwise <c>false</c>.
            /// </returns>
            protected override bool MoveNextCore()
            {
                this.index++;
                if (this.index >= this.data.Count)
                {
                    return false;
                }

                // Add one to keep index one based.
                this.resultCursor.MutableDataValue.ValidValue = this.data[this.index] + 1;

                return true;
            }

            /// <summary>
            /// The implementor should implement this method to reset the
            /// enumerator. If this method is called then the <see cref="M:Spotfire.Dxp.Data.DataRowReader.MoveNextCore"/>
            /// method should return the first row again when called the next time.
            /// </summary>
            protected override void ResetCore()
            {
                this.index = -1;
            }
        }
    }

Add Action Scripts to the Text Area

2010-03-11T21:11:32Z

A huge addition to the TIBCO Spotfire Developer license in Spotfire 3.1 is the ability to use a scripting language to add document-specific scripts to Action Controls in the Text Area. These scripts have access to the entire Spotfire C# API and can be executed in both Professional and the Web Player! This will open up an entirely new frontier in the development of sophisticated Spotfire Analytic Applications that behave exactly like the application author wants.

(This new functionality will be an especially welcome addition to DecisionSite power users who built a lot of Guides with JavaScript/COM in DecisionSite!)

IronPython is the scripting language that is supported with this new functionality. IronPython is Microsoft’s officially supported implementation of the Python language and interpreted through the .NET Framework. All the software needed to write and execute Python scripts is deployed with the Spotfire platform engine. Python is an easy to learn language, but in practice the majority of the code you will write behind Active Controls will require much, much less knowledge of the Python syntax and much, much more knowledge of the Spotfire C# API. If you do want to learn the details of the IronPython/Python syntax, Microsoft’s

http://ironpython.net/ site is a great place to start.

Watch the Spotfire Technology Network, this Knowledge Base and the User Community for a lot more information and examples on the power of IronPython scripting to solve many challenges, but the best place to get started is with the Using Scripts in the Text Area topic of the user manual. It takes you through the steps to build and deploy your first script along with a number of useful example scripts.

TIBCO Spotfire Administration Console

2010-03-11T20:25:57Z

With 3.1, the Spotfire Server adds a new web browser-based Administration Console. This tool is used to manage users and groups (unless users and groups are handled by an external mechanism such as an LDAP server), and to deploy Spotfire packages.

It can be reached by a standard web browser using the URL http://spotfireserver/spotfire/administration/, where spotfireserver is the hostname of one of your Spotfire Servers. If you are not running Spotfire Server on the standard port 80, you must also add the port number in the URL: http://spotfireserver:8080/spotfire/administration/. When prompted for a user name and password, use a Spotfire user who is assigned the Spotfire Administrator license. It does not matter which server in the cluster you use, changes made to one server will be stored in the Spotfire database and available to all servers.

The Administration Manager is still available through Spotfire Professional in order to manage users, groups, licenses and preferences but Spotfire package deployments are no longer managed there, they can only be managed through the web-based Administration Console. In this release, licenses and preferences can only be managed through Professional.

Below is a screenshot of the new web-based deployment mechanism:

Additional Web Player Mashup APIs

2010-03-11T19:47:45Z

The pre-Spotfire 3.1 JavaScript API for developing mashups includes functions for modifying the objects that are used in the Web Player, marking, filtering, changing page and applying bookmarks. That version of the API also included events that trigger when one of these objects are changed.

Many new APIs were added in 3.1 to make it easier for web developers to integrate with the new Property-driven analytic applications and to build mashups that are not hardcoded to a specific Spotfire DXP file or dataset.

Specifically, the new Spotfire 3.1 JavaScript APIs include the ability to:

Get, set, enumerate and listen for changes of:

Document Properties
Data Table Properties
Column Properties

Get the current value or state for the existing functionality:
- Marking
- Filters
- Pages
Enumerate and get values (to go along with the existing set methods) of metadata for the document:

Data Tables
Columns
Filtering schemes
Filters
Pages
Bookmarks
Markings
Analysis file

All of this should allow web developers to build even richer and more integrated applications that combine the unique capabilities of the Spotfire Web Player with existing web-based applications.

On-Demand Data Table Loading

2010-03-09T20:33:00Z

On-Demand Data Tables allow virtually unlimited amounts of data to be analyzed in a single Spotfire session. The key to this mechanism is that end user interaction with a Spotfire Analytic Application triggers a database to be re-queried based on the newly generated criteria and the returned table replaces an existing in-memory data set. The result is a seamless navigation experience.

Prior to Spotfire 3.1 the only user interaction that could be used as input was a single visualization marking and the resulting table had many limitations on how it could be transformed and worked with.

With Spotfire 3.1 almost all of those limitation have been removed, including:

Data Source

Parameterized and Stored Procedure Information Links can be used as data sources
An On-Demand Data Table can be the 1st (only) data table in an analysis
A new data format between the client and Spotfire Server decreases transfer time

Inputs

Multiple Markings can be used as input parameters, allowing for support of navigating independent star schema dimension tables and drilling into the fact table
Filterings can be used as input parameters, so no needless data is returned from the server
Custom Expressions (including Document, Data Table and Column Properties) can be used to define inputs

When combined with the new user input Property Controls this means that no input values have to be hardcoded or per session any more
The value passed to the query no longer has to actually be in the currently loaded data set

Input criteria can be specified as a range rather than a fixed list of items, so for example your query could return all values in the database between the minimum and maximum date that are marked in the “time dimension” regardless of whether those dates are marked or even in the data set
The number of categories per input can be restricted to limited to any given number, for example you could limit the number of “Countries” that can be selected by the end user to 3
Information Link prompts are optional
The definition of the input parameters and values can be edited at any time, without having to replace the entire data table

Transformation

Transformations, like pivot and normalization, can be applied to the resulting data table Rows, Columns, and calculations can be added to On-Demand Tables
Table Relations are not required so there does not have to be a join compliant relationship between the Information Link and the end data table

For more information, the online user manual has a complete description and set of examples for adding on-demand data tables and configuring where the input parameters come from and when they are triggered: http://stn.spotfire.com/stn/UserDoc.aspx?UserDoc=spotfire_client_help%2fadd%2fadd_loading_data_on_demand.htm

Heat Map with Hierarchical Clustering

2010-03-09T04:52:00Z

Spotfire 3.1 includes a heat map visualization, which is similar to a table or cross table except it shows colors instead of numbers and is space constrained to show all the information at once. When used with proper color management and sorted by attribute or by clustering results, it provides a powerful overview of your data. Heat maps used with clustering algorithms became very popular in biological research for visualizing massive microarray datasets to help decipher the relationship between genomic structure and function. But the Spotfire 3.1 heat map is flexible enough to visualize virtually any kind of data to uncover relationships and trends.

At first, the heat map can be a bit intimidating since it won’t necessarily reveal any interesting features of the data.

But by simply sorting by a column you can begin to see how the other columns generally relate to the sorted column.

The Spotfire 3.1 can automatically sort itself into groups of similar rows. The heat map is tightly integrated with a hierarchical clustering algorithm, and also has a special cluster visualization/navigation feature called a dendrogram to help navigate the results of the clustering.

Further, the clustering results can be navigated in a way that integrates with other visualizations. Here, the analyst can drag the pruning line (red dotted line) of the dendrogram to define the cutoffs for cluster groups, and that cutoff can dynamically drive (for example) the trellising of a line chart to reveal how the clusters appear in profile view.

The 3.1 heat map isn’t only for genomics research. It can be configured all the same ways as a cross table and help you find patterns in any numeric data. Here is an example of finding company stocks that behaved similarly over time:

Finally, the powerful coloring capabilities of Spotfire 3.1 open up many possibilities for using the heat map. You can create independent color schemes for different heat map columns, and even create your own dynamic coloring schemes based on custom expressions.

Combination Bar and Line Chart

2010-03-09T04:04:00Z

In response to overwhelming customer demand, Spotfire 3.1 includes a new visualization for showing bars and lines at the same time. Like other Spotfire visualizations it is highly configurable and interactive, it works seamlessly with the other visuals for drilling and showing relationships, and best of all it makes comparing trends easier for common cases like comparing actual figures to forecasts or quotas, or simply keeping track of a key reference while browsing other related trends.

Here are some other key features for this versatile plot …

Just click on the icons in the Legend when using Spotfire Professional and you can quickly change which trends are lines and which are bars …

You can choose to stack multiple sets of bars and put the side-by-side

Sort by one of the trends to see how well the others are following … in this case the x-axis is sorted by the bars representing the Health Care sector.

And finally, multi-y scaling means that trends can easily be compared from vastly different scales. Notice how each trellis panel has its own scale. Individual trends can also have their own scaling within the same frame.

Spotfire Integration with S+ and R

2010-03-09T04:03:00Z

Overview

Version 3.1 makes it easy to create Spotfire applications infused with computational methods written directly in S+ or R (* see footnote). This is how Spotfire can easily become the intuitive, interactive interface to custom, sophisticated applications that find patterns, detect outliers, predict trends, cluster groups, optimize outcomes, and much much more. Spotfire 3.1 gives users a great deal of control over the values and data sent off for calculation by S+ or R, and also provides creative ways to display and refresh the results within Spotfire clients. You can even make these powerful interactive applications available to web users by simply saving the application file to the Spotfire Library. Best of all, you can do all this with point-and-click configuration, so you won’t need to write a line of code other than any custom S+ or R script you want to leverage.

Benefits

Operationalize your analytic expertise. Here is the typical disconnect in today’s enterprise: One one hand, you have analytic experts know how to handle analytic challenges but are tied to arcane toolsets and it is difficult to rapidly scale out their work. On the other hand, business analysts have the passion to solve their domain problems but may lack sophisticated analytic methodology or access to the experts. Custom applications can be built to bridge this gap, but app building toolsets like C# and JAVA also demand time from specialized, constrained resources like IT groups. Spotfire 3.1 unleashes analytic power of your personnel by making it easy to configure reusable analytic applications that put the right amount control and flexibility in hands of business analysts in a scalable way without requiring a line of code.

A Simple Example: Loess Trend Smoothing

Here is a simple example of integrating an S+ script with a Spotfire application. The blue trend line is raw trend data loaded in Spotfire. The dotted red line is drawn from smoothing results calculated from an S+ script that utilizes a function called “loess”. The smoothed trend is configured to automatically recalculate every time the original trend data is filtered. (Click here to try using this application right now!)

Here is a little more detail on how this example is put together. The thing to remember is that these computationally sophisticated apps are configured, not coded. We start off with a simple trend line of values over time.

We register our loess algorithm a Spotfire “Data Function” by cutting and pasting S+ script right into Spotfire. The inputs and outputs required for this script are also described here. Registration is a one-time event. Once the script has been registered as a Data Function it can be referenced in as many different applications as you like.

Now that we have registered the Data Function, we can incorporate it into our application and map the inputs and outputs appropriately to our dataset. Now is also the time to decide if the calculation is refreshed manually by the user or automatically each time the input values are changed or filtered.

The loess results come back as a separate table of data. A new feature called “Line from Column Values” makes it possible to juxtapose the newly calculated smoothed values with the original data.

When the original data is filtered in Spotfire, the data is re-sent to the S+ engine, the a new smoothed trend is calculated, the new result is returned to Spotfire and refreshed visually, and all this happens automatically. To make this application available to web users, all you need to do is click File > Save As… > Library Item and you’re done.

Inputs and Outputs

For a given calculation Spotfire can send any number of single values, single columns or entire tables of data as inputs to S+ or R. Spotfire can also handle any number of single values, columns or data tables as return from S+ or R. It is even possible to return images generated by S+ and see them inside Spotfire.

In Spotfire 3.1 there are new user controls available within text areas that can change visualization settings or adjust variables in calculations. These same user-modifiable settings and variables can be used to determine input parameters to S+ or R scripts and functions. This gives builders a great deal of control in creating powerful applications that are easy to use.

When to Use the Local Adapter

When experimenting or prototyping with an S+ or R powered Spotfire application, you may not wish to constantly communicate with a Spotfire Statistics Services server. You can override Spotfire to use a locally installed S+ or R engine by installing the Local Adapter. This reduces load on server-based resources and gives you greater freedom to experiment. The Local Adapter is bundled with Statistics Services to enable local development, though the local S+ or R engine being called from the Adapter must be installed separately.

Requirements

Required to create the application shown in the example above: TIBCO Spotfire Server, TIBCO Spotfire Statistics Services, TIBCO Spotfire Professional & Metrics

Optional related products: TIBCO Spotfire Web Player for web based application usage, S+ desktop or Workbench for developing S+ scripts and functions, add-on modules that enhance S+ such as FinMetrics and NuOpt, TIBCO Spotfire Local Adapter (comes with Statistics Services), and R*.

* R is available under separate open source software license terms and is not part of the TIBCO Spotfire product line. As such, R is not within the scope of your license for any TIBCO Spotfire product. R is not supported, maintained, or warranted in any way by TIBCO Software Inc. Download and use of R is solely at your own discretion and subject to the free open source license terms applicable to R.

Color

2010-03-08T23:54:27Z

Overview

Spotfire 3.1 provides innovative new ways to color visualizations and tables, and provides new methods to manage and share the color schemes you use most often. Color in 3.1 can be configured based on static values, based on dynamic calculations that refresh as you manipulate the data, based on values that the user controls manually, or even based on relative definitions such as ‘top 5.’ You can easily update and propagate color schemes in your applications, and you can share schemes with your organization saving them to the Spotfire Library. Please read below for a full summary of color improvements in 3.1.

Rule Based Coloring

Often analysts want to values to jump out at them based on simple concepts like the ‘top and bottom 5’ or ‘everything over 100.’ Spotfire 3.1 makes it extremely easy to configure visuals to be colored in these ways. For example, to see top 5th percentile of values in a cross table, simply create a rule to color all values greater than the 95th percentile. See the sequential screenshots below that illustrate assigning blue to the top 5th percentile of values in a cross table.

You can apply more than one color rule at the same time. for example, You can set separate colors for the top 5 and the bottom 5 values at once.

You may also assign rules even if there are other coloring schemes in place. For example, you can choose a gradient color scheme to transition color from minimum to maximum values, but then create a rule to color the highest value a special standout color.

Coloring the Table

In Spotfire 3.1 you can set a color scheme for one or more columns in a Table view, making it easy to notice rows of interest. You can independently color different columns as needed. In the example below, two different columns are colored by two independent schemes.

Coloring the Cross Table

You can set a color scheme for one or more columns in a Table view, making it easy to notice rows of interest. You can independently color different columns as needed.
In the example below, each column has it’s own continuous gradient scheme to highlight the values that are close to or above quota.

Gradient Coloring and Segment Coloring

Gradient and Segment coloring modes are modes are used for coloring numeric ranges. Gradient coloring will gradually transition colors between the “anchor” points you specify, and segment coloring lets you set one fixed color for all the values in between two specified anchor points. You can set anchor points at fixed values or dynamic values like the average or median of a value range. You may add as many anchor points as you like. In the example below, a quintile color scheme is created by adding an anchor point for every 20th percentile in the data.

Here is a quintile gradient color scheme:

And here is a quintile segment color scheme:

Unique Value Coloring and Custom Palettes

You can set specific text values to a color. When you save and then re-apply this kind of color scheme, you have the choice to reassign specific colors to the same values, or simply apply the selection of colors to be reapplied in order. When creating color schemes where one color is assigned per value, you can choose to simply re-apply the color selection rather than specific assignments of colors-to-values. In this way you can create and use custom categorical palettes to apply to you categorical data instead of using the default Spotfire color schemes.

Saving and Reapplying Color Schemes

Spotfire 3.1 includes new ways to manage and share color schemes you create.

Save your color schemes to the Library so others can use them
When creating a new Spotfire plot, you can browse existing plots to use the color schemes you’ve already applied
If a color scheme that you use frequently changes, there is an easy way to update all the plots in a file that you want to be updated to the new scheme
You can still export color schemes to the file system and manage them on your own (available prior to 3.1)

Default Schemes

Spotfire has a helpful set of default color schemes to select from to help you get started …

Advanced Coloring with Custom Expressions and User Controls

If you set coloring anchors based on custom expressions, you can tie a color scheme to one or more values that the user determines on-the-fly. In this example below, the transition point between red and blue markers is determined by the slider control at the top of the page. This makes it possible for the user to adjust the colors to help certain features stand out more or less as needed. It also makes it possible for the user to adjust more than one plot at once. (For more on user input controls, see the article Introducing User Input Controls in 3.1)

Marking Improvements – Lasso Marking and Axis Marking

2010-03-08T23:20:27Z

Marking in Spotfire 3.1 has two significant improvements. “Marking” is what happens when you click on or drag a box around items in a Spotfire visualization. Marking can be configured to show you details about the data, to show the selected data in a different visualization, to retrieve related data On-Demand, to show related values in different plots, and more. So improved marking controls in 3.1 just means better ways to interact with your data.

First, Lasso Marking is a way to select visualization items by drawing a freehand area where everything inside will become marked. This is especially useful in the scatter plot and maps when you want to drill into collections of markers of interest, because they rarely fit neatly into the default rectangle-shaped marking areas. Grab precise clusters of markers, hug fitted curves, and enjoy the control.

In the sequential example below, lasso marking shows how it is possible to drill into the data points along the coastline by giving you freehand control to describe the exact marking area with your mouse.

Axis Marking is a way to easily focus on any markers within a range of values with a quick swipe of the mouse. Simply click and drag the mouse just outside the X or Y axis and you will see the marking area sweep across the plot for your selected values.

In the sequential example below, axis marking makes it possible to quickly focus on all the values above $2M.

Improvements to K-Means Clustering and Line Similarity

2010-03-08T23:19:18Z

When looking at a line chart in Spotfire you can use K-Means or Line Similarity tools to find clusters or patterns or matching trends respectively. In 3.1, Euclidian Distance has been added as an optional similarity measure to use when performing K-Means Clustering or Line Similarity calculations. Previously, K-Means and Line Similarity only supported correlation for finding patterns which meant that similarity of trends was judged based on the relative shape of the lines no matter how far apart they might have been by the Y-axis. Now that Euclidian Distance is offered, you can cluster or seek similar trends based on their nearness to one another in Y-axis space. See the illustrations below based on the Line Similarity tool:

We start with thousands of trends, but one particular trend is of interest and we want to find out which other trend line are most similar to the target trend.

Go to Tools > Options and run the Line Similarity tool. If you choose “Correlation” as a similarity metric, then similarity is judged based on the shape of the line. Here is what the plot looks like after filtering down to the top 10 results using this method.

But if you choose the new Euclidian Distance similarity option in 3.1 the top 10 most similar results to the target trend will be based on nearness rather than shape. You can see that the trends are not necessarily shaped exactly similar to the target trend, but they are located closely to the target trend.

Introducing User Input-Driven Analytic Applications

2010-03-08T20:29:00Z

New to the Spotfire 3.1 platform is the ability to add Property Controls to Text Areas. These easy-to-configure controls allow an application builder to effectively ‘guide’ the consumer to perform more meaningful analyses, like the ability to change visualization axes through the Web Player, perform ‘what-if’ analysis and pass inputs to Spotfire Statistical Services functions & On-Demand Data Loading. Available Property Controls include drop-down lists, list boxes, sliders , labels and text input fields. Configuration of these controls is UI-driven, without the need for any programming skills.

The process for creating a user input-drive analytic application is (in pretty much any order):

Define a Document, Data Table or Column Property
- There are shortcuts to performing this step in either step 2 or 3 below, otherwise it can be performed in the Data Table Properties, Column Properties or Column Properties dialog boxes under the Edit menu.
- Scope the variable as either being a document, data table, or column property.
- Properties have a name, description, data type, and default value.
Reference the Property in a Custom Expression
- The Custom Expression dialog box has been updated in 3.1 to provide for Properties to be inserted into expressions.
- The context menu (usually right-click) of the properties list allows you to add new properties.
- Replace any valid text in an expression with the contents of a property selecting the text to be replaced and the property and then pressing Insert Properties.
- See what will be passed to the data engine by looking in the textbox below the “resulting expression” label.
Configure a Property Control in a Text Area to assign its value(s) to a Property
- Add a Text Area visualization to a page and a Property Control to the Text Area.
- Pick a control type.
- Assign its output value to an existing Document, Data Table or Column Property (or create a new one).
- Where appropriate, configure where the valid input values come from (list of columns, values, fixed list, numeric range).

The following movie gives you a quick step-by-step overview of how you would author an user input-driven analytic application that allows an end user to switch the column that is used for the axis of a visualization:

Predictive Analytics with TIBCO Spotfire and S+ or R

2010-03-06T11:43:00Z

Last week we discussed an important new concept in TIBCO Spotfire version 3.1, scenario analysis. This week we will continue learning about new 3.1 features by discussing another type of analytics: Predictive Analytics. With TIBCO Spotfire version 3.1, you can communicate directly with S+ or R to execute scripts and functions. You can then return the data back into Spotfire as rows, columns, tables, and even values to properties, like Document Properties, Data Table Properties, and Column Properties...(read more)