Category Archives: TIBCO Spotfire

How Long to my ROI?

iStock 800433994 e1531505180465 How Long to my ROI?

Digital Transformation

Let’s block ads! (Why?)

The TIBCO Blog

Still Lost On New Industry 4.0 Keywords? Find Your Answers Here.

iStock 819365524 e1531417321940 Still Lost On New Industry 4.0 Keywords? Find Your Answers Here.


Let’s block ads! (Why?)

The TIBCO Blog

Create a Web Service Documentation Workflow Like Amazon

iStock 925557862 e1531256786480 Create a Web Service Documentation Workflow Like Amazon


Let’s block ads! (Why?)

The TIBCO Blog

Why More Developers are Turning to TIBCO

iker urteaga 246955 unsplash 960x557 Why More Developers are Turning to TIBCO


Let’s block ads! (Why?)

The TIBCO Blog

What’s a “Data Function” and Why it is Such an AI Enabler?

Many uses of artificial intelligence or machine learning are well served using high-quality visuals depicting discoveries or predictions. To achieve this efficiently, techniques and algorithms should be well integrated into the tool’s visualization architecture. Leland Wilkinson’s “The Grammar of Graphics” (Springer, 2005) is an inspiration for many open source and proprietary visualization systems (including portions of TIBCO Spotfire). Most implementations concentrate on the book’s primary guidance around the graph algebra and mapping aesthetics to data. That concept works nicely to provide a very wide variety of visualization capabilities and options.

The book also notably addresses how analytics interact with visualizations. Rather than considering a visualization as simply the display vehicle for the output of a routine the book instead argues an approach where analytics perform “in the service of the visualization”. This subtle, but very important distinction allows rich visualizations capable of multiple layers of analytic content without having to perform a lot of data manipulation outside the visualization tool to force fit complex analytic result data structures into a single table.

This architectural concept is demonstrated in the TIBCO Spotfire “data function” feature. Data functions are the place where visualizations and analytics interact in the product. Any visualization can have multiple data functions performing calculations to support it and each data function can produce any number of tables of output. The resulting variables in those tables can be used for position and aesthetics across the visualization. This goes far beyond typical tools offering predictive analytics integration that simply augment a table with additional output columns.

The results can be seen in examples that use different algorithms such as route optimizations, contours, and inventory predictions that all come together into a single visualization (a map) in a very development-friendly way. Further, since UI controls are directly tied to algorithm specification parameters, data functions simplify the authoring experience by data scientist for their dashboard users.

Optimization What’s a “Data Function” and Why it is Such an AI Enabler?The above example uses R algorithms executed in Spotfire’s built-in TERR engine; however, other statistics or ML tools can similarly be used. The latest release of Statistica now also supports defining data functions via Statistica workflows. In the example below, thresholds and warning levels required for a statistical process control chart are calculated using a Statistica workflow. These calculations happen dynamically—the execution is initiated by user actions—such as selecting a range of data to evaluate.

SPC What’s a “Data Function” and Why it is Such an AI Enabler?The Spotfire data function is a true gem that is very practical for developing modern, data-driven, visually rich AI applications.

Let’s block ads! (Why?)

The TIBCO Blog

Data Science and Machine Learning: From Academics to Economics

iStock 898079502 e1530207517150 Data Science and Machine Learning: From Academics to Economics

In my previous post, I shared our excitement that TIBCO’s analytics solutions received the highest score for Production Refinement in Gartner’s Critical Capabilities for Data Science and Machine Learning Platforms 2018 report. To recap, the Production Refinement use case is heavily influenced by the Delivery, Model Management, Performance and Scalability, and Machine Learning critical capabilities. Furthermore, Gartner stated that production refinement is where Data Scientists spend most of their time.

Most organizations are keenly aware that analytics is the fuel for a successful digital transformation. There are all sorts of studies and statistics that suggest that digital leaders significantly outperform digital laggards by a significant margin (e.g. 5x higher revenue growth) with only a slight increase in technology spending (3.5% vs. 3.2% of revenue). So, given the fact that many organizations have the access to voluminous amounts of data and there are a plethora of algorithms available, is there a secret ingredient that most organizations are missing?

Let’s use Netflix as an example. On a daily basis, Netflix streams 250 million hours of video around the world (190 countries) to 98 million paying subscribers. Everytime you click, browse, watch, pause, rewind, stop, start, rewatch, all of this data is collected and analyzed to personalize your next experience. We are all keenly aware that companies like Netflix, Amazon, and Google use recommendation systems and make next best offer recommendations; but did you know that Netflix even personalizes the graphics for each viewer segment? For the show “Stranger Things”, did you know that I may find the show in the “TV mysteries” category and you may find it in the “sci-fi thriller” category? Netflix has around 2000 “taste communities” (aka customer segments) and for the popular series “Stranger Things”, Netflix’s algorithms apply 12 tags to capture the intricacies of how different people relate and react to the show; in other words, there are 2000 different customer segments each with its own unique viewing experience and 12 different intricacies for this particular show! Now, that must be a lot of AI, machine learning, and analytics being used to optimize your viewing experience.

Netflix’s runs on Amazon Web Services (AWS) and consists of a whole lot of infrastructure, microservices, and data science to stream shows around the world. Netflix’s ability to orchestrate people, processes, and analytic technologies in real time has allowed them to monetize their data and gain a significant competitive advantage. This is the secret ingredient that makes Netflix who they are. Now you don’t have to be Netflix to pull this off.

We have many customers around the world who are managing hundreds and perhaps thousands of models. Whether they be in banking, insurance, manufacturing, energy, or health care we hear time and time again, the ability to put quickly test and deploy analytics to production systems is what leads to competitive advantage. Through automation and orchestration, one customer in the banking industry reduced the time they spent on developing and deploying models by 50%! That 50% more time they can be innovating and optimizing customer experiences.

When organizations think about data science, their mind quickly races to the algorithms and mathematics. However, this is not where the real value lies. The real value lies in being able to monetize their data through analytics which create insight. To deliver the insight, companies need to increase focus on the deployment, management, and monitoring analytic models. In order to move from academics to economics, organizations need to focus on production refinement, this is the secret ingredient which will lead to competitive advantage.

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

1 Harvard Business Review. What the Companies on the Right Side of the Digital Business Divide Have in Common. By Robert Bock, Marco Iansiti, and Karim R. Lakhani. January 31, 2017.

2CNet. ‘Stranger Things Addict? Here’s how Netflix sucked you In. October 23, 2017.

3How Netflix works: the (hugely simplified) complex stuff that happens every time you hit Play.

Let’s block ads! (Why?)

The TIBCO Blog

Spotfire 7.13 is Now Available with 17 Cool New Features

We are very excited to announce that we have just released Spotfire 7.13. We have delivered 17 new features and enhancements across visual analytics, data access, data wrangling, API’s, and administration. I am going to highlight one cool feature from visual analytics and data access & wrangling themes. Make sure to check out the full list at What’s New in Spotfire and the community wiki.

Visual Analytics

A very cool feature added on the visual analytics theme is automatically adjusting zoom sliders. Visualizations with zoom-sliders now auto-zoom when the data changes (e.g, when filtering) and when sliders are at the end of their range:

Auto Zoom GIF Spotfire 7.13 is Now Available with 17 Cool New Features

Data Wrangling

Continuing on our “Edit Everything” theme, we are very excited to tell you that with Spotfire 7.13, you can add columns to a data table using the Spotfire Business Author web client as well. No more having to switch to Spotfire Analyst to make fixes when you discover issues with your joins.

Add Column GIF Spotfire 7.13 is Now Available with 17 Cool New Features

Data Access

TIBCO Cloud Spotfire supports Apache Spark SQL and Microsoft HDInsight Hive, offers support for Amazon EMR and we now have added a timeout setting to the Cloudera Impala connector allowing for longer query run times.


Spotfire 7.13 makes it possible to trigger execution of Automation Services jobs from external applications using a REST API. The Web Service (SOAP) APIs now use a OAuth2 based authentication/authorization mechanism. We have also simplified the procedure for building .NET extensions for Spotfire. Beginning this release, it is now possible to ship a bundled solution, containing several Spotfire packages, as a single distribution file (.sdn).

For a full list of features release in Spotfire 7.13, please visit us on the web, community and check out the video.

Let’s block ads! (Why?)

The TIBCO Blog

A Culture of Analytics: Why Amazon & Netflix Succeed While Others Fail

iStock 855485708 e1530043005312 A Culture of Analytics: Why Amazon & Netflix Succeed While Others Fail

When it comes to advanced analytics projects, it often seems like success stories are the exception rather than the rule. Many organizations would like to emulate the data-driven culture displayed by leaders like Amazon, Google, and Netflix, yet have trouble with operationalizing analytics. And there’s broad agreement that analytics and ‘big data’ projects fail at very high rates — more than half the time.

If I had to pick one fundamental reason why, I’d say that it’s because analytics projects are irreducibly complex and multi-faceted: they typically have many moving parts, more than a software project for example. So there are many opportunities to stumble along the way. Projects often fail right at the start, when data is off limits or hard to find. And project goals may shift rapidly: often it’s not known whether the data will confirm hypotheses at all, or yield actionable insights, or support predictive models. There may be problems of scalability, of integrating with operational systems, of model accuracy, and so on. And, in the end, there is the stubborn problem of how to deploy complex workflows and models, often requiring a tedious manual conversion from data scientist code into real-time scoring.

As a result, analytics projects can quickly become overwhelmed with technical details. I believe that an agile, business-driven approach can help ensure success.

Analytics initiatives are often viewed solely through a technical lens, resulting in situations where organizations get lost in the weeds agonizing over statistical models, database structures and platform infrastructure. Amazon and Netflix are able to use data in innovative ways not just because they are technically advanced, but also because they’ve created a “culture of analytics” that pervades every aspect of their business.

Here are four key considerations for executing an analytic strategy that puts the needs of the business first:

  1. Have a purpose. While this might seem obvious, the reality is that many companies lose their way with analytics because they focus first on technical specifications and not enough on a tangible business objective. In effect, they put the technical cart ahead of the business horse.

Let’s step away from analytics for the moment and imagine another type of project — constructing a building. Would you purchase the materials and hire general contractors, plumbers, and electricians before you had a clear understanding of the building’s purpose? Of course not.

Yet all too often, this is exactly how organizations proceed with analytics deployments. They start building infrastructure and hiring data scientists and evaluating technologies (this or that database? Hive or Spark SQL?) long before they’ve specifically defined the business problems and opportunities that can be addressed using analytics. A technology deployment, no matter how seamless or advanced is not going to magically transform your organization into a data-driven powerhouse like Amazon, especially if its purpose is vague and poorly defined.

To avoid costly mistakes, business end users must be part of the analytics strategy from day one. These are the stakeholders who can weigh in on the use cases that have the most to gain from big data and provide a realistic picture of how analytics would/could fit into the applications and processes that they rely on every day to do their jobs.

And start with just one analytics project, with high business potential, and readily available data. Then figure out what technology you need.

  1. Link “insight” to action. What does it really mean to solve a business problem? In the analytics world, it’s become common to assume that the best way to solve a problem is to provide more information, or “insight,” about it. But that’s the equivalent of having a meeting to address a specific issue and “resolving” it by scheduling another meeting. Insight, to be of any value, must be predictive in nature and – this is the important part – drive action quickly, seamlessly and automatically.

It’s this last step that trips up many organizations. Imagine a company that wants to optimize its Q2 sales. The organization might start by attempting to use analytics to better predict sales volume. But telling the sales team that volume is expected to drop in Q2 and even providing insight into why it might do so is not enough. Predictions should launch tangible actions that sales and marketing can take to turn things around.

Amazon, Google, and Netflix are masters at transforming insight into data-driven action. For example, Amazon uses big data to automatically customize the browsing experience for its customers based on their past purchases, and optimize sales. Netflix seeks to directly impact customer behavior with data-fueled recommendations and, more recently has used data to spawn the successful creation of original content that they’re confident audiences will like.

Connecting analytics to actual results demands “high resolution” data and predictive analysis that prompts actions within purchasing, sales, lead generation – whatever the business objective may be. But one final piece is needed to make this all simple, and seamless: integration.

  1. Push analytics to business end-points. Most business and customer-facing stakeholders within an organization don’t know Hadoop from NoSQL from Apache Hive. Nor should they. Analytics infrastructure is highly technical and not easy for non-engineering types to interact with and use.

Successful analytic cultures find ways to push outputs to and integrate seamlessly with, the go-to applications that real-world sales, marketing, procurement and other business-level decision-makers use on a regular basis. These include tools like, Marketo, and Zendesk. A “touchpoint” approach to analytics concentrates on how and where analytics will realistically be utilized and ensures that lessons learned are integrated into these applications — a critical part of connecting insight to action.

  1. Create feedback loops. Finally, an analytics strategy is never “done.” This is another reason to avoid getting stuck in the weeds by laboriously perfecting statistical models over many months. The analytic climate is always changing for most organizations as new data sources become available, and as business opportunities, challenges and priorities evolve.

Organizations should, therefore, focus on a flexible analytics infrastructure that is not only able to meet a number of diverse requirements across the enterprise, but also one nimble enough to adapt quickly to the needs of the business. This is where the “agile” movement in software development can serve as a useful example: instead of aiming for a “perfect” solution, focus on rapid development of fresh models that can be pushed quickly into production. Then, monitor performance with A/B testing and use “feedback loops” to continually test, refine and improve analytic applications and begin the cycle again. In this way, organizations can rapidly and continuously get actionable data out to the users who need it, even as needs evolve.

The most advanced analytics-driven companies in the world give their employees remarkably free access to their codebases. They encourage safe experimentation with open access to source-code, rigorous code reviews, clearly-defined metrics of success, and instantaneous feedback loops.

Many companies want to be able to emulate data darlings like Amazon, Google, and Netflix. But those organizations have succeeded because they have wisely integrated analytics into the very fabric of their business. Before jumping into the deep end with highly complex technologies and advanced algorithms, companies just getting started should first address low-hanging fruit and build analytic applications that are valuable to actual business users. Don’t let the perfect database or latest and greatest statistical model get in the way of achievable results.

(This blog entry is based on an article that originally ran in insideBigData.)

Let’s block ads! (Why?)

The TIBCO Blog

Developing Your Digital Transformation Strategy

iStock 928819758 e1529607181424 Developing Your Digital Transformation Strategy

Digital Transformation

Let’s block ads! (Why?)

The TIBCO Blog

API-Led Architecture is the New API Strategy

iStock 913589018 e1529600929833 API Led Architecture is the New API Strategy


Let’s block ads! (Why?)

The TIBCO Blog