Category Archives: Data Mining

Pentaho 9.1 is available!

October 19, 2020 Pentaho

Pentaho 9.1 is available

It’s that time of the year! A new release is available!

Go get EE through the support portal, and CE in the usual place!

Main features

· Google Data Proc Support

· Catalog Steps in Spoon

· New Upgrade Utility

· And a bunch of consolidation stuff:

o 20+ Continuous Improvements

o 10+ Platform Updates

o 200+ Performance/quality bugs

Google Data Proc

You can now access and process data from a Google Dataproc cluster in PDI. Google Dataproc is a cloud-native Spark and Hadoop managed service that has built-in integration with other Google Cloud Platform services, such as BigQuery and Cloud Storage. With PDI and Google Dataproc, you can migrate from on-premise to the Google Cloud.

You can use PDI’s Google Dataproc driver and named connection feature to access data on your Google Dataproc cluster as you would other Hadoop clusters, like Cloudera and Amazon EMR. See Set up the Pentaho Server to connect to a Hadoop cluster for further instructions.

§ What’s New

‒ New Hadoop driver

‒ AEL-Spark support

§ Version:

‒ Google Dataproc – 1.4 (Ubuntu 18.04 LTS, Hadoop 2.9, Spark 2.4)

§ Benefit

‒ Enables processing large data sets in Google Data Proc clusters

‒ On-premise data movement/migration

§ Hadoop Driver supports the following:

‒ Multi-cluster

‒ HDFS

‒ Hive

‒ PMR Hive

‒ Oozie

‒ Sqoop

‒ Hadoop Job Executor

‒ Pig

‒ Parquet / Avro / ORC

§ VFS support for GCS

§ Hbase is not supported

Lumada Data Catalog steps for PDI

Lumada Data Catalog lets data engineers, data scientists, and business users accelerate metadata discovery and data categorization, and permits data stewards to manage sensitive data. Data Catalog collects metadata for various types of data assets and points to the asset’s location in storage. Data assets registered in Data Catalog are known as data resources.

You can use the folllowing four new PDI steps to work with Data Catalog metadata and data resources within your PDI transformations:

· Read Metadata

Search Data Catalog’s existing metadata for specific data resources, including their storage location.

· Write Metadata

Revise the existing Data Catalog tags associated with an existing data resource.

· Catalog Input

Reads the CSV text file types or Parquet data formats of a Data Catalog data resource that is stored in a Hadoop or S3 ecosystem and outputs the data payload in the form of rows to use in a transformation.

· Catalog Output

Encodes CSV text file types or Parquet data formats using the schema defined in PDI to create a new data resource or to replace or update an existing data resource in Data Catalog.

New Upgrade utility

• Current Scope:

‒ Scope only 9.0 to 9.1 (coming later: will extend to 8.3 LTS)

• Reliable upgrades and rollback

‒ Initial environment check to detect product component and will only upgrade what is there

‒ White list to persist customization

‒ Will persist all plug-ins across upgrade

‒ Automatically whitelist all database driver jars

Compatibility Updates

This:

Other improvements:

This:

§ Data Integration

‒ S3 Multipart Upload now allow configurable part sizes (PDI-16606)

‒ MongoDB Plug-in now allows PLAIN credentials for LDAP integration (PDI-17228)

§ Dashboards / Reporting

‒ 10-100x performance improvement for certain large slices and roll-ups for Mondrian Cubes (JIRA Link)

‒ Option to remove/hide the filter panel when used in a dashboard (ANALYZER-2270)

‒ Count and Count Distinct Summary on currency fields uses the default format (PIR-699)

‒ Admins can now customize the template(s) used for exporting to PDF and Excel (ANALYZER-12)

§ Platform

‒ Passwords stored in the BA Server config files and repository are now encrypted (BISERVER-3497)

‒ Users are now able to change their own password (BISERVER-13699)

-pedro

Let’s block ads! (Why?)

Pedro Alves on Business Intelligence

Quantitative and Qualitative Data: A Vital Combination

October 18, 2020 Sisense

We live in a world of data: There’s more of it than ever before, in a ceaselessly expanding array of forms and locations. Dealing with Data is your window into the ways organizations and data teams tackle the challenges of this new world to help their companies and their customers thrive.

Almost every modern organization is now a data-generating machine. As soon as a company’s systems are computerized, the data-generation engine starts up. When these systems connect with external groups — customers, subscribers, shareholders, stakeholders — even more data is generated, collected, and exchanged. And, as industrial, business, domestic, and personal Internet of Things devices become increasingly intelligent, they communicate with each other and share data to help calibrate performance and maximize efficiency. The result, as Sisense CEO Amir Orad wrote, is that every company is now a data company.

Smart use of your data can be the key to optimizing processes, identifying new opportunities, and gaining or keeping a competitive edge. But are you paying attention to all of your data? Do you have the means to handle every kind of data? And can you take full advantage of the insights it can reveal? Your answers will depend on whether you can gather and analyze both quantitative and qualitative data. Let’s consider the differences between the two, and why they’re both important to the success of data-driven organizations.

Quantitative and Qualitative Data: A Vital Combination

Digging into quantitative data

Most commonly, we think of data as numbers that show information such as sales figures, marketing data, payroll totals, financial statistics, and other data that can be counted and measured objectively.

This is quantitative data. It’s “hard,” structured data that answers questions such as “how many?” or “how often?” It’s in an organized format, usually rows and columns, and is stored as a relational database, in which there are relationships among these rows and columns.

All descriptive statistics can be calculated using quantitative data. It’s analyzed through numerical comparisons and statistical inferences and is reported through statistical analyses.

As quantitative data is always numeric, it’s relatively straightforward to put it in order, manage it, analyze it, visualize it, and do calculations with it. Spreadsheet software like Excel, Google Sheets, or traditional database management systems all mainly deal with quantitative data. These programs and systems are great at generating basic visualizations like graphs and charts from static data. The challenge comes when the data becomes huge and fast-changing.

Why is quantitative data important?

Quantitative data is often viewed as the bedrock of your business intelligence and analytics program because it can reveal valuable insights for your organization. These numbers show performance, efficiency, reach, market share, revenue in, and expenses out. This is fundamental information when it comes to understanding how your organization is doing.

Additionally, quantitative data forms the basis on which you can confidently infer, estimate, and project future performance, using techniques such as regression analysis, hypothesis testing, and Monte Carlo simulations.

These techniques allow you to:

See trends and relationships among factors so you can identify operational areas that can be optimized
Compare your data against hypotheses and assumptions to show how decisions might affect your organization
Anticipate risk and uncertainty via mathematically modeling

Consequently, using quantitative data, you can make strategic and tactical decisions that will benefit your organization and drive growth.

What are the problems with quantitative data?

Despite its many uses, quantitative data presents two main challenges for a data-driven organization.

First, data isn’t created in a uniform, consistent format. It’s generated by a host of sources in different ways. To access it effectively, you need to organize it and clean it to avoid mistakes, oversights, and repetitions that could compromise the integrity of your data and, by extension, the accuracy of your insights.

The solution to this problem is to ensure that your BI and analytics platform can handle as many sources and forms of data as possible, both on-premises and on the cloud, so that you don’t neglect or miss any of it.

Second, and more challenging, is the fact that not all the information you generate and collect is structured, quantitative data. As data is produced from a growing array of sources, quantitative data may be as little as 20% of all the data available to most organizations. If you focus your efforts entirely on quantitative data, you’ll overlook a huge amount of valuable information, and your insights and decision-making could become distorted as a result. You need a solution that can access and analyze the other 80%: qualitative data.

Exploring qualitative data

Qualitative data is unstructured, meaning it’s not in a predefined format. In its raw state, it comes in a variety of forms: text, social media comments, phone call transcripts, various logs files, images, audio, video, and more.

Often, the information in qualitative data is a categorical variable. This means it describes the characteristics or qualities of data units, such as “what type,” “which category,” “who” (or “which persona”). These create a picture of the context and environment in which the data can help build an understanding of feelings, opinions, and intentions. That’s because qualitative data is concerned with understanding the perspective of customers, users, or stakeholders. This type of data is often collected through less rigid, measurable means than quantitative data. Examples include comments, recordings, interviews, focus group reports, and more, typically received in the natural language of the informants.

Qualitative data is much more subjective, “soft” information than quantitative data. However, advanced analytics can now identify and classify this information and transform it into findings that lead to game-changing insights for organizations.

As the sources of data continue to proliferate, an increasing proportion of it is unstructured and qualitative. It’s more complex, and it requires more storage. Traditional methods of gathering and organizing data can’t organize, filter, and analyze this kind of data effectively. Advanced technology and new approaches are needed. What seem at first to be very random, disparate forms of qualitative data require the capacity of data warehouses, data lakes, and NoSQL databases to store and manage them.

Qualitative data benefits: Unlocking understanding

Qualitative data can go where quantitative data can’t. For example, it’s the gateway to sentiment analysis — understanding how users, customers, and stakeholders think and feel, as well as what they do. Techniques that focus on qualitative data, such as content analysis and narrative analysis, enable you to:

Analyze text to find the most common themes in open-ended data such as user feedback, interview transcripts, and surveys, so you can pinpoint the most important focus areas
Better interpret feelings or perceptions about your organization, product, services, processes, or brand, to help identify what changes or innovations would be most effective

Having access to this information gives you a new dimension of insights on which to base decisions and determine the tactical and strategic direction of your organization. With qualitative data, you can understand intention as well as behavior, thereby making predictive analytics more accurate and giving you fuller insights. You can analyze and learn from the large volume of unstructured data to ensure that your data-driven decisions are as solid as possible.

For example, Skullcandy, the manufacturer of headphones, earbuds, and other audio and wireless products, uses predictive and sentiment analysis to understand customers better. This enables the company’s product development team to explore opportunities for reducing returns and warranty requests for new products before they’re even released and get reactions from customers about current products. Skullcandy uses these insights to hone its new products based on what customers say they like and dislike, so that it can offer more attractive products to its target market.

Getting the most from qualitative data

Qualitative data eludes the rigid categorization and the storage limitations of traditional databases. It provides the raw material for information that is more varied and harder to organize than structured, qualitative data. Making sense of and deriving patterns from it calls for newer, more advanced technology.

Natural language processing (NLP), involving machine learning, is how your BI and analytics platform can understand the meaning of unstructured data such as emails, comments, feedback, and instant messages. It enables search-driven analytics that allow you to process complex NLP, because you can ask the system questions using natural, everyday language. Plus, you can receive visualizations and embed insights into external apps, even if what you’re asking for is relatively complex from an analytical point of view.

Furthermore, systems powered by AI and augmented analytics continuously learn what people choose to do with data. Algorithms identify patterns in the data, and these make search results faster, more accurate, and more complete. You can integrate AI analytics tools with other interfaces like Amazon Alexa and chatbots, drawing on these technologies’ impressive developments in NLP, too.

With all this potential to find new insights, it’s no surprise that 97% of business leaders say their businesses are investing in big data and AI initiatives.

Better together: Working with qualitative data and quantitative data

Quantitative data is the bedrock of your BI and analytics. It provides a solid foundation on which you can build data visualizations and insights that are fundamental to your organization.

With that foundation in place, qualitative data enables you to take your analytics further, get more understanding from a fast-growing array of sources, and power your decision-making with data from even more perspectives.

You have to be able to work with both kinds of data in order to unlock the most comprehensive insights. Together, they can help set the stage for game-changing pivots and innovations that can keep your organization at the head of the pack.

overcoming challenges in nlp Blog Banner 770x250 1 Quantitative and Qualitative Data: A Vital Combination

Adam Murray began his career in corporate communications and public relations in London and New York before moving to Tel Aviv. He’s spent the last ten years working with tech companies like Amdocs, Gilat Satellite Systems, and Allot Communications. He holds a Ph.D. in English literature. When he’s not spending time with his wife and son, he’s preoccupied with his beloved football team, Tottenham Hotspur.

Let’s block ads! (Why?)

Blog – Sisense

5 Principles for Sustainable Innovation with Cloud Manufacturing

October 17, 2020 TIBCO Spotfire

Reading Time: 3 minutes

Though the rise of Industry 4.0 in the last few years has encouraged manufacturers to digitally transform, the latest global events have made it more of a priority than ever before. It’s become clear that manufacturers need to change the way they implement operations, use data, engage with customers, and create an overall more sustainable and resilient ecosystem. And all this can be done with a cloud-based strategy.

The truth is that many manufactures are not adequately prepared for disruptive events. Disruptions are caused by a number of different reasons:

Advancements in technology can cause strong disruptions in the way of doing business.
Disruptive innovative competitors can affect your business; therefore, having the right insights can avoid investing in the wrong ideas or products.
Customers’ tastes and behaviors continuously influence the company’s strategies forcing them to adapt production to be able to quickly react.
Economic volatility leading to changes in customer behaviors. Think of the automotive industry during this global pandemic; if you’re locked at home you don’t need access to a car.

The good news is that digital transformation is still top of mind for most manufacturers in many areas. Trends that have been found include:

40% of manufactures will use IoT sensor data to diagnose issues and reduce downtime, which requires predictive maintenance¹
50% of new IT infrastructure will be deployed outside of data centers¹
35% growth in digital twins to create a virtual factory²
20% of G2000 manufactures will employ IoT, ML, and AI to automate decision making³

The pandemic pushed us towards the future of manufacturing by almost five years. Before many companies had already started or had plans to start a digital transformation. Suddenly, a new unforeseen critical factor was added: staying in business.

Since the pace of digital transformation won’t slow down, the question is if today’s manufacturing industry is agile and innovative enough. And more importantly, is it resilient enough to survive the next disruptive event and outpace the competition?

The future of manufacturing leverages the concepts of cloud manufacturing to create a sustainable ecosystem to navigate through the next major event by successfully implementing these five core principles or 5Cs:

Connected
Collaborative
Cloud-centric
Customer focused
Continuous improvement

In order to achieve the 5Cs, you need to implement a cloud-based strategy that utilizes the following:

Integration to support SaaS providers and an API-first strategy.
Low-code approach to speed up the time to market, create newer services, and innovate faster.
Power back to the people to empower all levels of users and create more synergy with processes and technology.
Hybrid architectures to decide what data to migrate and when.
Scaling and deployments to support out-of-the-box scaling to increase service availability and speed up the release cycle of new services.
Processing at the edge to allow more edge IoT, 5G, and serverless computing patterns.

Cloud also enables modern event-driven responsive architecture. With a responsive architecture, you can process critical factory events and support your operations with immediate actions. You gain the necessary agility for the so-called “new normal”.

Here are a few of our customers who have implemented cloud manufacturing:

Campari is one of the largest spirits beverage producers in the world. Since 1995, Campari has made 27 acquisitions so it needed to integrate with all different kinds of systems. The beverage producer had the pressure to minimize the time to market. Only by moving to the cloud was Campari able to gain the necessary business agility to sell the newly acquired products as fast as possible on the market and to sustain future acquisitions.

Cosentino is a family-owned business founded 40 years ago in Spain and active in mining hills for granite and selling unfinished slabs to stonemasons. It completely transformed its business when it went digital to become a leader in high-end kitchen and baths finishing. But with the expansion, Cosentino had to face challenges to scale operations and keep running many different business processes. By integrating their global IT systems in the cloud and adopting a cloud-native architecture, a customer is now able to see in real-time if a product is available in two to three days compared to three to six months.

The future of manufacturing leverages the concepts of cloud manufacturing to create a sustainable ecosystem to navigate through the next major event. Click To Tweet

Manufacturing isn’t going away any time soon. But, if manufacturers want to remain competitive in this new environment, they have to look towards digital transformation, specifically with cloud. This will give manufacturers the necessary sustainability, agility, and resiliency to be prepared for the next disruptive event.

Learn more about how TIBCO enables manufacturers of the future and how these capabilities can help your business.

Let’s block ads! (Why?)

The TIBCO Blog

“We Belong to a Bigger Community” Hispanic Heritage Month Q&A

October 14, 2020 TIBCO Spotfire

Reading Time: 3 minutes

As a part of TIBCO’s Diversity & Inclusion initiative for Hispanic Heritage Month, we are spotlighting employees who are of Hispanic heritage, asking them what makes them proud of their heritage and what they plan to do to embrace their identity in the future.

TIBCO: Tell us about yourself

Bruna Wells: My name is Bruna Wells and I’ve been at TIBCO for one year and five months as a Partner Account Manager in the LATAM region, based in São Paulo, Brazil. I’m from a big family—a sister of seven brothers. I’ve been with my boyfriend-now-husband Billy for 21 years. I am the mother of Caio (14) and Luana (2). At TIBCO, I participate in the BLAAC and Women of TIBCO employee resource groups (ERGs).

TIBCO: This year’s Hispanic Heritage Month theme is “Be proud Of Your Past, Embrace The Future.” What is your Hispanic heritage?

BW: My mother is Chilean, but I was born and raised in Brazil. I spent all my adult life traveling to different Latin American countries absorbing the different cultures, way of life, and people’s histories, which have shaped my personality. So I consider myself a Latin American citizen.

TIBCO: What about your heritage’s past are you most proud of?

BW: The Latin American resilience. We have faced a lot of barriers to building our own identity and independence. I cannot be more proud and thankful for all our ancestors who always persevere and succeed in light of the regional, political, climate, and economic challenges we face.

TIBCO: What ways will you “embrace your future” during Hispanic Heritage Month and beyond?

BW: Most of the countries in Latin America celebrate Independence Day in September so this is an opportunity to talk with our families and friends about our expectations for this uncertain future that we face together. When thinking about my colleagues in TIBCO, I’m a part of the leadership team of the BLAAC ERG where we are planning several activities to make our community feel embraced and represented.

TIBCO: In what ways do you feel the Hispanic communities have helped shape the world?

BW: Our high energy is incomparable. We are known worldwide for our charisma, our hospitality, our parties, and our optimism, always showering others with lots of good food and drinks. In any country that you visit in Latin America, I can guarantee you will return home feeling fulfilled and loved.

TIBCO: What does being a part of the Hispanic community mean to you?

BW: It’s diverse but in a unique way. Everyone in Latin America has a particular history, but we belong to a bigger community from different countries that understands ourselves, our culture, and our past. The identity and genuine curiosity of each person’s personal history is priceless.

TIBCO: What are the benefits of celebrating our shared history and cultures throughout TIBCO?

BW: Working at TIBCO means that what you are and where you come from matters. I love hearing my colleague’s personal histories and feel inspired to share mine. When I know a person’s background I start to admire them and feel more connected to them. And by contributing, I expect people to feel more connected to me too.

TIBCO: What is a Hispanic tradition that you wish to pass down, that your parents have passed down to you?

BW: Our faith. My parents raised me to believe in God and be grateful for the small things like our food and our bed, in addition to praying to sustain and fill our spirit. In a country where 90% of the population considers themselves religious, this is a tradition that I am already passing to my children. It represents a lot to me since my mother and father instilled it in me.

TIBCO: Any other thoughts you would like to share related to being part of the Hispanic community within TIBCO?

BW: Unfortunately my grandmother that lived in Chile passed away during this pandemic, and it was really hard to say goodbye to her and be there for my mother virtually. In the past several months, we had to solve a lot of challenges together and through that, a lot of our Hispanic heritage came alive again. This will always be present in our lifestyle as well as her. During this process, I could count on my team at TIBCO to support me and understand the situation.

Working at TIBCO means that what you are and where you come from matters. I love hearing my colleague’s personal histories and feel inspired to share mine. Click To Tweet

During the month of October, join us in celebrating Hispanic Heritage Month. Are you a TIBCO team member? To get involved and join an ERG today, visit TIBCO Connect for more details!

Let’s block ads! (Why?)

The TIBCO Blog

6 Ways TIBCO Can Help You Sustainably Innovate

October 12, 2020 TIBCO Spotfire

Reading Time: 3 minutes

In late September, we were really excited to have so many of you join us for our first all-digital TIBCO NOW. The TIBCO NOW Connected Experience was full of insightful keynotes, engaging customer conversations, awards, as well as a slew of product releases. From now through October 9, you can still register and access all the great content on demand. Catch up on all the great keynotes, breakout sessions, and perspectives from innovative leaders from around the world.

Keep reading for six ways TIBCO can help you sustainably innovate.

Cheers To Our TIBCO Lighthouse Award Winners!

Each year, we are excited to recognize our customers that put TIBCO technologies to use to address some of the world’s most challenging opportunities. Learn about how these industry leaders are pushing the boundaries of innovation to achieve digital transformation and improve society for all. If your company has successfully utilized TIBCO technology to transform your enterprise, fill out the nomination form so we can share your story during next year’s TIBCO Lighthouse Awards at TIBCO NOW 2021.

Introducing the New TIBCO Cloud Integration “Big Basin Release”

You can now connect and monitor your apps wherever they reside. That means seamlessly watching all your remote apps that might be in the cloud, or on-prem or in a hybrid environment, on one new unified dashboard. There’s also a marketplace to easily connect to every available app. For more information, please refer to the product documentation and see this demo.

Get Real-time Awareness & Take Action with TIBCO Cloud Data Streams

In a real-time world, you have the tall task of analyzing and responding to changing conditions—without any delays. Whether it’s business-critical data from customer transactions, or streaming data from manufacturing, supply chain, or the IoT, TIBCO Cloud^™ Data Streams can help. TIBCO Cloud^™ Data Streams allows you to work with real-time data to stay up-to-the-moment and ahead of the curve. Read this whitepaper to learn more about how TIBCO Cloud Data Streams is democratizing operational intelligence with streaming business intelligence simplicity.

Boost Business Agility With the TIBCO Responsive Application Mesh

Aligning technology choices with the needs of the business can seem impossible, yet it’s a puzzle many organizations are faced with everyday. We’ve developed The TIBCO Responsive Application Mesh to help our customers evolve rigid, legacy application architectures into highly flexible digital platforms using modern cloud-based technologies that adapt quickly to rapidly changing business requirements. In this series of white papers, you’ll learn how it can boost agility for your business.

CXO Insight of the Month: How to Overcome Cloud Data Silos

A key to a strong organization is being nimble and leveraging data to make strong decisions. Cloud software deployments have been a catalyst for many organizations in this regard, but many are seeing the unintended challenge of multiple cloud instances. Each cloud deployment is an opportunity for data to be siloed. The good news is that it can be fixed with these keys to breaking down data silos within your organization.

Become A Data Science Superhero—Hone Your Skills And Get Rewarded

Data scientists are as rare as superheroes these days. We’ve identified six essential skills to help you become a powerful data scientist and a superhero to your organization. By joining and taking in our Data Science Superhero Challenge, you’ll be eligible to win some great prizes. Start your origin story by joining the TIBCO League challenge.

Sustainable business practices and data-centric tools are critical components for long term financial performance as well as societal impact. TIBCO’s mission, highlighted at this year’s TIBCO NOW, is to give organizations what it takes to re-invent themselves now and into the future. We call it “Sustainable Innovation”. And we’ll continue to deliver on that promise through insights, tools, and expertise to help you transform. In the meantime, binge-watch content to your heart’s content through October 9.

Let’s block ads! (Why?)

The TIBCO Blog

Overcome the Most Common Roadblocks to Digital Transformation

October 10, 2020 TIBCO Spotfire

Reading Time: 3 minutes

Your digital business is constantly changing: customer demands are shifting, as are the means of production and operation. To support effortless experiences, new business opportunities, and real-time intelligence, your business needs to be agile. However, transforming into a digital business that enables agility is not easy. You have a lot of legacy architecture and a culture that’s just not ready to change. Fortunately, you now have two resources to help you overcome any roadblocks you might be facing in your digital transformation efforts.

Both Gartner and TIBCO have recognized the most common roadblocks to digital transformation and have laid out manageable road maps of how to overcome them. Using both the Gartner report (talked about below) and the TIBCO solution (the TIBCO Responsive Application Mesh blueprint), you can become a truly agile, digital business, regardless of your current technology debt or any other hurdle that may be preventing you from becoming a digital organization.

To begin, Gartner explains three application architecture trends that you can use today to facilitate transformation, modernize your application portfolio, and deliver new capabilities. The three modern application architecture trends that they recommend you adopt to help transform are:

Mesh App and Service Architecture (MASA)

First, you must build a mesh app and service architecture (MASA) that provides fundamental architectural capabilities that enable applications to multi-channel experiences. It responds rapidly to digital business demands such as agility, shared business capabilities, security, and resiliency. As digital business depends on a flexible application portfolio that enables effortless experiences, supports new ways of doing business, and makes the most of new technologies, now is the time to implement a MASA in your organization.

API Platform

Your organization needs APIs that enable access to the services that implement your business capabilities and data. However, It is not enough to just create APIs; you must also manage and govern your APIs and pull them together into a platform of shared business capabilities. You need a full lifecycle management API platform and strategy to design, build, publish, advertise, and manage your APIs.

Event Processing

The fast pace of modern business requires that organizations improve their real-time awareness and decision making. Event driven architectures have proven to be a differentiator for those businesses that have taken the time to learn and deploy them. They deliver intelligent insights gleaned from event processing solutions detecting and responding to patterns in business operations, often in real time. Event processing is an old concept that has never really taken off in many enterprises, mainly due to its apparent complexity. However, embracing this type of architecture in a phased way can greatly speed up your digital transformation efforts.

TIBCO Can Help You Transform into a Digital Business

According to Gartner, to build business agility and transform into a digital business you need to start with three modern application architectures: MASA, an API-platform, and event processing. These provide the foundation for rapid innovation—one that is API-led and event-driven to maximize connectivity and take action on events in the business as they occur in real-time.

To help your organization take advantage of these modern application architectures, TIBCO has devised an easy to use blueprint, called the TIBCO Responsive Application Mesh. This blueprint can help you efficiently build a technology foundation that will enable agility, regardless of your current infrastructure and technology debt. Meaning – if you use the TIBCO Responsive Application Mesh, you don’t have to throw away what you already have. We will help you transform no matter what roadblocks you may face.

To support effortless experiences, new business opportunities, and real-time intelligence, your business needs to be agile. Click To Tweet

Digital businesses are constantly demanding new application capabilities. Thus, application leaders should work to modernize their application infrastructure with the latest trends to enable these capabilities. Read this report from Gartner and learn about the TIBCO Responsive App Mesh to learn how you can implement modern application architecture trends to facilitate transformation regardless of your current situation.

Let’s block ads! (Why?)

The TIBCO Blog

How to Build a Flexible Developer Documentation Portal

October 8, 2020 Sisense

Developing analytic apps is a bold new direction for product teams. The Toolbox is where we talk development best practices, tips, tricks, and success stories to help you build the future of analytics and empower your users with the insights and actions they need.

When creating a resource and community to help developers get the most out of your product, it’s important to empower them to contribute to developer documentation and not just have all your content coming from product or tech writers.

If you haven’t read how we overhauled our developer portal recently, check out our prior conversation with Moti Granovsky, Sisense’s Head of Developer Relations. In this second part, we will take a deeper dive and share how we built a new system from the ground up to both deliver great information and be flexible enough to allow for continued evolution and growth.

Let’s kick off our journey into the rebuild by understanding what our requirements were and how we went about meeting them.

How to Build a Flexible Developer Documentation Portal

Building instead of buying

Shruthi Panicker: Why did we choose to build the portal on our own from scratch and not just use one of the many great products out there?

Moti Granovsky : There are great products out there focused on developer documentation, which are used by many companies successfully. When we set about rebuilding the portal, we knew we wanted engineers to contribute to our developer documentation. We also wanted to auto-generate some of the content, especially API references (which are very dry; there’s no “writing” necessary). Our product has so many APIs (rich and large) that writing and maintaining them is very time-consuming. We didn’t want tech writers to waste time repeatedly updating tables and tables of parameters.

We wanted to invest our time in generating content that is more engaging and valuable, like tutorials, open-source demos, cool features, and so on.

Make your requirements specific

SP: What were you asked to do?

MG: Our requirements were, in essence:

The platform should allow us to separate styling from content. We wanted to make sure that unlike the old website, it looks great, matches the Sisense brand, and keeps on looking great even as more content is added. This means not only did we want to white-label and design the content, we wanted to separate the layout and design from the content. (Remember, the people who are writing documentation are not necessarily experts at visual design.)
The format should be accessible to developers (aka Markdown-based). Besides an UI-based editor, some content management systems have their own specific language to programmatically format content. However, oftentimes nobody knows enough about these languages, and they’re not very intuitive either. Instead, we looked at the most common way for developers to write content — Markdown. It is used by Git itself and all Git providers. Even readme files for projects are written in Markdown. Every developer knows this syntax, and it has this benefit of separating design from content. Markdown content will also allow us to auto-generate content. If you look around, most API tools like Swagger, for example, have the ability to generate Markdown content.
Our customers should be able to contribute to docs (eventually). If a customer finds a typo or a mistake or they want to add some detail, we want to make that possible. For that, we should be able to store the content somewhere public, such as a GitHub repository. Markdown makes this possible since it’s text format and can be easily shared in GitHub. But we need the content to be separate from the website itself (something that is publicly available). If we were to write HTML pages and put them on a server, then a customer can’t contribute to that. But if those pages are generated from something that is publicly accessible like a GitHub repository, then we can solve this challenge.
We should go beyond just the documentation. We wanted the new website to be a true developer portal — a hub of knowledge and information for developers with which they can find all the resources they need. We want to make sure that, in the future, it includes a blog, playground, links to community, etc. that are not necessarily documentation. We want to create a home for developers where they can come and find anything that they need to do their job.

We looked at products in the market, and none of them met all of these requirements combined. Additionally, they were relatively costly because they include hosting, but documentation is generally static content so it is cheap and easy to host. There was no reason we couldn’t build it ourselves since we have the technical know-how to do it in-house.

sisense blog Embedded Trends 20191218 bl blog banner How to Build a Flexible Developer Documentation Portal

Choose the right framework

SP: So what tools did you decide to use to build the new portal?

MG: We found out some companies (and a huge shoutout to the developer relations team at Okta whose approach to developer docs inspired us) have taken this different approach and built their own websites using different tools. There are frameworks that can generate websites from Markdown. We looked at different ones and picked one that fit the Sisense technology stack (in terms of languages that developers are familiar with. Since most of our developers are JavaScript developers, we wanted one that used that and not Python, for example).

We found one called VuePress that did exactly what we needed. It has all the flexibility in the world. It is an open-source framework built and used by the Vue.js project. It allowed us to build exactly the type of website we had in mind in terms of look and feel while avoiding a lot of the effort involved compared to starting from scratch.

We invested effort in creating a great design and refactoring a lot of the existing content we had into Markdown from the old format. However, this process let us build the new portal from the ground up exactly the way we want without any compromises.

While it took a lot of effort initially, in the long term it will all pay off. First, the hosting is very cheap. Second, if we have to add new content, we don’t have to design anything. It will only be as expensive as writing that text. Anyone can just write text in a notepad and have beautiful content up on the website.

New portal, new options

SP: What can users expect to find in the new portal that they couldn’t in the old one?

MG: As a result of the new design, there are a few big changes in the new Sisense developer portal:

Separation of API reference and documentation: Developers can find a dedicated API reference section that is well organized and structured. They can quickly access the exact information they need. It is a lot more detailed and a lot more standardized than what we previously had, and they don’t have to scroll a bunch of text to access what they need.
More code samples right within the documentation: Code can be easily copied with a copy code button. It is nicely formatted, and the syntax is highlighted according to the language, making it much easier to read.

portal1 How to Build a Flexible Developer Documentation Portal

A lot more cross-referencing and links than we used to have: When something is mentioned, it is always linked right there, or you can find the relevant link at the bottom of your page. You can navigate through and gain all the knowledge to accomplish a task without breaks in the thought process.

Improved search: You can quickly see search pages and headers within pages. You can quickly jump into specific areas within a page.

portal2 770x299 How to Build a Flexible Developer Documentation Portal

Responsive mobile experience: We also optimized all pages for mobile consumption.
New and refactored content: Much of what’s been added to the portal is entirely new, plus we refactored a lot of existing content, revalidating and updating it.
Broad range of content types: We added a lot of external references to our GitHub account, webinars hosted on Vimeo, and a playground built in conjunction with the developer portal.

Overall, the content now is not just a block of text but part of an ecosystem of knowledge that developers can utilize. Everyone has a different learning process. Some prefer to read, some prefer to see a demo, some like to watch a video while others prefer to just dive right in and learn through experimentation. We are trying to provide different ways to help no matter what a person’s approach to learning is.

In addition, we also have features that cover:

Developer release notes and release notes for the website itself so you can track what has changed
More about the DevX team, why we exist, and and how to get in touch with us
Ability to reach the playground, blog, and forums from the portal directly

In the future, we are looking at adding a developer-focused blog within the portal directly. We are also continuously working on more video content and documentation for some of the APIs that aren’t covered yet.

How you can improve your own documentation

SP: Lastly, how can an organization decide if building its own developer documentation site ground up is the right approach for them?

MG: Take into consideration these points to decide if this approach is good for you:

You need engineering capability to build the website.
You need to make sure that the people who are writing documentation are comfortable with the format. For example, if the people writing documentation in your organization are technical writers who are comfortable with Markdown, then great, use this approach. Or maybe they want to learn, then that is great as well. If they do not want to learn, they may need another platform that is what-you-see-is-what-you-get.
Make sure you have the ability in terms of IT to host and maintain the site. If you can’t assure that it will be up globally and available 24/7, then you are going to have problems. Since this is the largest interface with the largest surface area, fault or downtime will have a huge negative impact.

It all boils down to capacity and need. If your current documentation website works and customers are happy with it, and all you need to do is to enhance and update the content, then do that. Don’t waste your time and money on building something new. But if you have a lot of room for improvement like we did, and you want to use the opportunity to upgrade and overhaul the user experience, then this is a viable solution that should be considered.

build developer documentation portal blog cta banner 770x250 1 How to Build a Flexible Developer Documentation Portal

Shruthi Panicker is a Sr. Technical Product Marketing Manager with Sisense. She focuses on how Sisense can be leveraged to build successful embedded analytics solutions covering Sisense’s embedding and customization capabilities, developer experience initiative and cloud-native architecture. She holds a BS in Computer Science as well as an MBA and has over a decade of experience in the technology world.

Let’s block ads! (Why?)

Blog – Sisense

Top 3 TIBCO4Good Sessions at TIBCO NOW

October 7, 2020 TIBCO Spotfire

Reading Time: 2 minutes

TIBCO NOW 2020 was a fantastic example of a work hard, play hard mindset in action with numerous learning opportunities as well as happy hours, tea vs. coffee debates, and yoga breaks. The TIBCO4Good^™ initiative embodies this spirit as well, taking our hard work and expertise and empowering employees to engage in rewarding, inspiring community initiatives.

The mission of TIBCO4Good is to help communities solve meaningful human challenges with technology, contributing software, training, expertise, and strategic guidance to partner organizations and communities in need. Below are some of our favorite TIBCO4Good stories that are making that vision a reality, as shared during TIBCO NOW.

TIBCO4Good Sessions at TIBCO NOW:

COVID-19 Visual Analytics Hub: In March, the TIBCO data science team built a hub of visual and predictive analytics to help communities track the coronavirus current outbreak status, growth trajectories, rate of spread, effects of interventions, and healthcare resources. At TIBCO NOW, the team demonstrated how this interactive Spotfire application pulls together data from various sources from around the world, unites them, and transforms them with data science to make predictions in real time.

To see other use cases where visual analytics, data science, and data management are put into action to solve hard business problems, watch this TIBCO NOW session on Hyperconverged Analytics in Operations with TIBCO Chief Analytics Officer Micheal O’Connell. Also, join Mr. O’Connell and global health leaders for a lively panel discussion on how healthcare organizations can utilize analytics to manage the impacts from the pandemic.

TIBCO GatherSmart & Dartmouth College: Using TIBCO GatherSmart™, an AI-powered monitoring solution for businesses and communities to control the further spread of COVID-19, Dartmouth College is working to create a responsible environment for those returning to work and to learn. Dartmouth will leverage TIBCO’s powerful analytics tools to organize its multiple-sourced COVID-19 data, enabling the institution to act quickly and limit exposure on campus. Join this TIBCO NOW fireside chat with Dartmouth College CIO Mitch Davis and TIBCO CTO Nelson Petracek, talking about preparing for and thriving in a COVID-19 era.

City Harvest London: City Harvest London’s mission was never in doubt — feeding the hungry by collecting and transporting unused, quality food. But COVID-19 nearly shut down a critical service at a time when food insecurity became more dire than ever. Collaborating with TIBCO Labs and TIBCO4Good, City Harvest built a cloud-based platform for connecting supply and demand, optimizing logistics, and successfully scaling to meet unprecedented demand. Watch this TIBCO NOW session to hear City Harvest CEO Laura Winningham and TIBCO SVP for Connect Randy Menon discuss the successful City Harvest London and TIBCO partnership.

Collaborating with TIBCO Labs and TIBCO4Good, City Harvest built a cloud-based platform for connecting supply and demand, optimizing logistics, and successfully scaling to meet unprecedented demand. Click To Tweet

Get Involved!

Also, coming soon TIBCO4Good is teaming up with our Microsoft friends to help map the most vulnerable communities around the world. Together we are partnering with a wonderful organization called Missing Maps, aiming to put 200 million people “on the map” by 2021.
Here at TIBCO, we recognize the importance of giving back and enjoy putting our passion for innovative technologies to good use. To get involved in any of our ongoing TIBCO4Good projects, head to tibco.com/tibco4good to learn more.

Let’s block ads! (Why?)

The TIBCO Blog

You Can Deploy and Monitor Your Integrations Anywhere with TIBCO

October 5, 2020 TIBCO Spotfire

Reading Time: 3 minutes

In a rapidly changing business environment, two tenets are critical to maintaining resiliency and unleashing future innovation: flexibility and real-time insights. As business operations are executed more by software and data rather than manual, paper-based processes, these tenets must extend to your application architecture and the integration applications that connect it together.

With today’s modern cloud-based application architectures, there are so many environments to choose from in which to deploy your integration applications: traditional on-premises, across various infrastructure and platforms-as-a-service (IaaS/PaaS) as well as containers-as-a-service (CaaS), serverless environments, and even lightweight edge devices.

With all of these deployment options available, it is vital that your integration solution gives you maximum flexibility to deploy your integration applications to the environments that best meet your business and technical needs.

In fact, in a recent survey conducted by Pulse.qa on behalf of TIBCO Software, when asked, “What 3 features are most important when considering an application and data integration solution?” The top feature identified was “Flexibility to deploy across a wide range of environments”, chosen by nearly 60 percent of respondents (see chart below).

You Can Deploy and Monitor Your Integrations Anywhere with TIBCO

A guiding principle at TIBCO is to give our customers a wide variety of technology choices, and that extends to our TIBCO Cloud Integration integration platform-as-a-service (iPaaS). Not only can you use TIBCO Cloud Integration on your choice of either the AWS or Microsoft Azure cloud platforms, but you also have the flexibility to deploy the integration applications you create with it into the environments of your choice, such as those mentioned above as well as TIBCO Cloud. To learn more about the flexibility provided by TIBCO Cloud Integration, we invite you to read our recent whitepaper, “10 Reasons Why TIBCO Is the Leader for Connecting Modern Application Architectures and Why that Matters to You”.

View and Monitor Your Integration Applications from a Single Location

This flexibility of being able to deploy anywhere doesn’t provide much value if you can’t obtain real-time insights into how your integration applications are performing. You need to be able to monitor your integrations across your chosen deployment environments at all times. Since the connections between applications and data on your digital platform underpin your business processes, integration applications that don’t perform as required (or fail to perform altogether) can have a significant negative impact on your business.

As announced at this year’s TIBCO NOW event, the TIBCO Cloud Integration – Big Basin release features hybrid monitoring capabilities, which enables you to view and monitor integration applications from a single location, regardless of deployment. Already available for monitoring TIBCO Cloud Integration – Connect applications, users are now able to additionally monitor remote Integrate (powered by TIBCO BusinessWorks™) and Develop applications (powered by TIBCO Flogo®) within the TIBCO Cloud Integration dashboard (see below). This creates a single monitoring experience for all applications regardless of the deployment environment.

Hybrid monitoring is enabled by the new TIBCO Cloud Integration – Hybrid Agent, a lightweight program that you run locally to facilitate communication between remote apps and TIBCO Cloud Integration. Downloadable within the TIBCO Cloud Integration dashboard, the TIBCO Cloud Integration – Hybrid Agent securely communicates information on registered remote apps, such as monitoring statistics, app status, and other app details. For more information, please refer to the product documentation.

Having these insights readily available at your fingertips within a single view ensures that your business stays connected. You can see a quick demonstration of this feature in the video below.

[embedded content]

TIBCO Cloud Integration – Big Basin release features hybrid monitoring capabilities, which enables you to view and monitor integration applications from a single location, regardless of deployment. Click To Tweet

In today’s fast-paced world, flexibility and real-time insights are critical. The TIBCO CloudIntegration Big Basin release gives you both. Learn more about TIBCO Cloud Integration and try it for yourself for 30 days at no charge.

Let’s block ads! (Why?)

The TIBCO Blog

To Improve Data Quality, Stop Playing the Data Telephone Game

October 3, 2020 TIBCO Spotfire

Reading Time: 3 minutes

Do you remember when you used to play the telephone game with other children? You know, the game where the first person in a chain whispers a phrase to the second, then the second repeats it to the third, and on until the last person repeats it back to the first.

It was such a joy to laugh about how what began as “The sun is in the sky” somehow transformed into “It is fun to eat pie” as the phrase passed from one friend to the next.

Further laughs quickly followed when each person in the chain shared their whispered phrase with all, allowing everyone to see what went wrong and where.

The Data Telephone Game

Interestingly, over the past thirty years, data management has adopted the same telephone game formula, copying data from one database to the next, with many stops on the journey.

Take the classic enterprise data warehouse process as an example:

The data starts as transaction records stored in a transaction system’s database
Next it moves from the source system to a staging database
From staging it moves to a data warehouse
With subsets of that data further advancing for storage within satellite data marts
Many of which soon feed individual Excel files resident on laptops

Or the more recent cloud data lake paradigm:

Source data from devices are consolidated on edge databases
This edge data is then copied into a cloud data lake for further analysis
Additional data from transaction systems might also be added to the lake
And to inject historical context, warehouse data might also be copied into the lake

Conceptually, these data management best-practices provide the opportunity to improve data quality by applying selected value-add transformations at various steps. But with so many rigid links in the chain, this data version of the telephone game can often inadvertently turn “sky” into “pie.” The business impact of this quality problem produces anything but childhood chuckles.

How Big is this Repeated Copying of Data Problem?

Just how much data is getting copied? In its Worldwide Global DataSphere Forecast 2019-2023, IDC estimates that for every terabyte of net-new data, over six additional terabytes of copied data is generated via replication and distribution. That is a lot of opportunities for “sky” to become “pie.”

Three Ways to Stop Playing the Telephone Game

IDC’s numbers, when combined with everyone’s telephone game experience, suggest trying a different approach to improve data quality. Here are three common-sense things organizations might consider.

Copy Less, Virtualize More – Data Virtualization is a proven method for integrating data without physically copying it. This will substantially reduce the transformation errors and entropy inherent in typical multiple-copy, data warehouse, and data lake deployments. Beyond fewer copies, data virtualization directly improves data quality via metadata-driven syntactic and semantic transformations and enrichments that standardize datasets and encourage reuse. Everyone is on the same page. And when things change, as they inevitably do, it’s a lot easier to modify centrally managed metadata definitions than it is to modify multiple distributed ETLs and database schemas.
Share Reference Data Everywhere – Reference Data Management improves data quality by enabling organizations to consistently manage standard classifications and hierarchies across systems and business lines. This lets them achieve needed consistency and compliance without extra copies. And by adding data virtualization as the distribution method, organizations can easily share and reuse reference data held in one virtual location.
Think Data Domain, Not Database Technology – Today, there are lots of cool, fit-for-purpose database technologies. But “new and exciting” doesn’t necessarily translate into “high business value.” Instead, think about the most valuable data domains. For example, if customer excellence is your competitive advantage, then focus on improving quality within the customer data domain. Master Data Management is the key to success in this case, allowing organizations to ensure data integrity within selected data domains such as customer, employee, product, and more.

Data Virtualization is a proven method for integrating data without physically copying it. This will substantially reduce the transformation errors and entropy inherent in typical multiple-copy, data warehouse, and data lake deployments. Click To Tweet

Stop Playing the Data Telephone Game

Let’s leave the telephone game to the kids. Instead, improve your data quality by executing the three common-sense recommendations above with TIBCO Unify. To learn more, talk to TIBCO and our partners.

Let’s block ads! (Why?)

The TIBCO Blog

Category Archives: Data Mining

Pentaho 9.1 is available!

Main features

Google Data Proc

Lumada Data Catalog steps for PDI

New Upgrade utility

Compatibility Updates

Other improvements:

Quantitative and Qualitative Data: A Vital Combination

Digging into quantitative data

Why is quantitative data important?

What are the problems with quantitative data?

Exploring qualitative data

Qualitative data benefits: Unlocking understanding

Getting the most from qualitative data

Better together: Working with qualitative data and quantitative data

“We Belong to a Bigger Community” Hispanic Heritage Month Q&A

6 Ways TIBCO Can Help You Sustainably Innovate

Overcome the Most Common Roadblocks to Digital Transformation

How to Build a Flexible Developer Documentation Portal

Building instead of buying

Make your requirements specific

Choose the right framework

New portal, new options

How you can improve your own documentation

Top 3 TIBCO4Good Sessions at TIBCO NOW

TIBCO4Good Sessions at TIBCO NOW:

Get Involved!

You Can Deploy and Monitor Your Integrations Anywhere with TIBCO

View and Monitor Your Integration Applications from a Single Location

To Improve Data Quality, Stop Playing the Data Telephone Game

The Data Telephone Game

How Big is this Repeated Copying of Data Problem?

Three Ways to Stop Playing the Telephone Game

Stop Playing the Data Telephone Game

Recent Posts

Categories

Archives