Unlocking Big Data Potential On-Prem and in the Cloud
TIBCO and Leidos partner to build modern DevOps in the big data community
TIBCO Spotfire Data Science® is an enterprise-class cloud platform for teams to collaborate on big data and analytics. As a long-standing partner with Leidos, the company’s platform is a component of the Leidos Collaborative Advanced Analytics & Data Sharing (CAADS™) framework. The collaboration enables an end-to-end scalable data analytics solution that addresses some of the most prominent challenges facing data scientists today.
Tackling these Data Challenges
1. Fuzzy Matching: Approximate string matching leads to bad data and inaccurate assertions. Our advanced data manipulation capabilities include machine learning, complex joins, and the ability to do imprecise matching. By outputting two files, one where every record is faithfully reconciled against every defined attribute and one where there’s imprecise matching that needs to be reconciled, data scientists can ensure errors are detected and eliminated to avoid frustrations common to big data analysis when records are joined incorrectly.
2. Analysis Paralysis: This challenge manifests itself in two ways — continuous testing with no outcome and solely relying on preferred, comfortable methods of testing that create a blind spot for potentially better tests. Our machine learning addresses both challenges. Analysts who would normally spend months trying to determine which regression is appropriate, or those who would spin their wheels using a mismatched method for the job, can simply run all of the regressions and work with the best outcome to move the project forward. The easy-to-use functionality is the key to eliminating workflow barriers to better outcomes.
3. Siloed Workflows: Silos can apply to data sources as well as research resources. Effective data analysis depends on bringing disparate data and diverse thinkers to a shared workspace. Our platform provides teams with the ability to collaborate on analysis even though they may be separated by time and space. Capturing institutional knowledge and aggregating data in one location is needed to solve important, high-impact challenges.
We’ve made it our mission to help our customers overcome these challenges and work more efficiently. And the CAADS framework is the difference maker — by integrating data from multiple sources, spinning up analytics projects rapidly, and simplifying the process of developing predictive models, CAADS has delivered actionable insights in healthcare and biomedical research, for applications as diverse as disease outbreaks and space flight.
Dr. Ryan Weil, principal scientist at Leidos, describes CAADS as a force multiplier. Referencing work done with the CDC Weil reports, “The end-to-end, fully scalable, ‘no code’ approach shaved the amount of time it took to do the analysis by a factor of six. Significantly shortening the analysis process allows our team to get incremental data products out quickly.” This is especially critical when analysis results impact public health. “Data too late won’t do,” continues Weil. “Accurate and timely information to the people who can act on it is essential for the best outcome.”
TIBCO Spotfire is an important component of the data ecosystem leveraged by CAADS users, addressing the data challenges listed above. First, it has in-database data preparation steps that include complex joins and matching. Next, it has all of the classic machine learning algorithms that can be built and tested in parallel on even the largest datasets. Finally, it enables sharing and searching of analytics assets within teams and between organizations.
As the product manager for Spotfire Data Science, I can confidently say that my team is proud to have contributed to the CAADS platform. Leidos has done really groundbreaking work with CAADS — they’ve really made a difference in health policy and disease control. And as a result, they’ve driven innovation in our analytics products, especially by demanding custom algorithms that proved the extensibility of the system.
What’s Next?
TIBCO is taking the partnership to the cloud. Working to translate their legacy of success to the on-demand delivery system, TIBCO and Leidos are eager to bring the benefits of secure, compliant cloud deployment to their customers.
Driven by new requirements continually emerging from private and public agencies across the research spectrum, including NASA, the Center for Disease Control (CDC), and the Centers for Medicare and Medicaid Services (CMS) — to name a few — Leidos is at the epicenter of innovation. By creating a web-based platform for data science, they’ve been able to leverage the flexibility of Amazon Web Services to spin up an entire big data infrastructure in an afternoon, ready-made for exploration and analysis.