• Home
  • About Us
  • Contact Us
  • Privacy Policy
  • Special Offers
Business Intelligence Info
  • Business Intelligence
    • BI News and Info
    • Big Data
    • Mobile and Cloud
    • Self-Service BI
  • CRM
    • CRM News and Info
    • InfusionSoft
    • Microsoft Dynamics CRM
    • NetSuite
    • OnContact
    • Salesforce
    • Workbooks
  • Data Mining
    • Pentaho
    • Sisense
    • Tableau
    • TIBCO Spotfire
  • Data Warehousing
    • DWH News and Info
    • IBM DB2
    • Microsoft SQL Server
    • Oracle
    • Teradata
  • Predictive Analytics
    • FICO
    • KNIME
    • Mathematica
    • Matlab
    • Minitab
    • RapidMiner
    • Revolution
    • SAP
    • SAS/SPSS
  • Humor

4 Data Quality Trends I Observed During Recent Data Governance Events

October 24, 2017   Big Data

Over the past five weeks, I’ve attended and presented at four different events, including the Strata Data, Data Governance Financial Services, and Information Management MDM & Data Governance conferences in the New York metro area and ASG EVOLVE in Washington, DC.

What’s clear to me… data governance is still at various forms of maturity across organizations, but also is important as ever, if not more so.

Data Quality in Data Governance

Why is data quality such an important part of data governance programs? Consider this quote from the recent report from Harvard Business Review, “only 3% of the Data Quality scores in our study can be rated ‘acceptable’ using the loosest-possible standard.” That’s not only telling of the importance of data quality, but it’s scary!

blog banner eBook Global Locator 4 Data Quality Trends I Observed During Recent Data Governance Events

My presentation at these conferences focused on the intersection of data quality and data governance. I talked about different aspects of both, including data profiling (understanding of what you have), data matching to create a single view, assigning data stewards and definitions, defining policies including data quality rules, and so on.

Data Lineage

Data lineage is all about understanding where data came from and how it was changed along the way.

There were lots of nods from all of the audiences when I talked about data lineage. You can’t just understand where the data came from a source perspective (this database table, that mainframe file), think about if it was joined with other data sources, aggregated, transformed or something else along the way. You need to have data lineage at the field or column level.

This is a hard problem. We’re doing this now with our data integration tool DMX-h, at the field level, publishing an API to get to the lineage and integrating with tools such as Cloudera Navigator and Apache Atlas.

blog Cloudera Navigator 1 4 Data Quality Trends I Observed During Recent Data Governance Events

Key Data Elements

Let’s get back to data quality. During the presentation, I talked about identifying key data elements, assigning data stewards, and creating data quality policies on these data elements to create data quality dashboards.

Key data elements are not in the tens of thousands, in fact, in one presentation I saw this month said they originally identified over 21,000 then reduced it by 85%! Comparatively, most organizations we work with have 100-200, maybe even less, to start with.

Combining Business and IT Data Knowledge

The people that “know the data” are in the business, not necessarily in IT. In my presentation, I gave an example from a Proof of Concept we did a few years ago.

We were doing the POC with IT, and while profiling the data we came across a state field that was MH. This is obviously not a US state, but when IT went to the business, MH was used for a person that worked in Massachusetts and lives in New Hampshire. When I mentioned this during my presentations and asked how many people here have data like this, there were a lot of “yes’s” and head nods.

The lesson here: the data stewards must be in the business.

4 Data Quality Trends Happening Now

I talked with a lot of people at these conferences. What I learned:

  1. Everyone confirmed data quality is a key part of data governance. In fact, you can’t have good governance without data quality
  2. Companies are at various maturity with regards to measuring and improving the quality of the data
  3. Some companies are already measuring their data quality, while many are just getting started
  4. “Boiling the ocean” doesn’t work. Many said their organizations tried this and always failed. Start small, operationalize it, learn, show success, then move on or expand.

Think about how to get started within your organization. We obviously can help.

Our new eBookhighlights success stories of organizations who use our global address validation tools to support Data Quality and business intelligence.

Let’s block ads! (Why?)

Syncsort + Trillium Software Blog

data, During, Events, Governance, Observed, quality, recent..., trends
  • Recent Posts

    • NOT WHAT THEY MEANT BY “BUILDING ON THE BACKS OF….”
    • Why Healthcare Needs New Data and Analytics Solutions Before the Next Pandemic
    • Siemens and IBM extend alliance to IoT for manufacturing
    • Kevin Hart Joins John Hamburg For New Netflix Comedy Film Titled ‘Me Time’
    • Who is Monitoring your Microsoft Dynamics 365 Apps?
  • Categories

  • Archives

    • February 2021
    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
© 2021 Business Intelligence Info
Power BI Training | G Com Solutions Limited