Category Archives: Business Intelligence

Notes from 2017 MIT CDOIQ Symposium: CDO’s, and Data, go Mainstream

During the conclusion of this year’s annual MIT Chief Data Officer and Information Quality Symposium (MITCDOIQ) held July 12-14 in Cambridge, MA, the organizers noted that the focus of the event was at last shifting from the rise and role of the CDO to a focus on the key, strategic business initiatives undertaken by the CDO and their organization.  But this shift was evident right from the start of the event – one featured session called this the process of “Shifting from Survival to Success.”

Front and center were key topics around the business value of data:  building out the data infrastructure and democratizing data, the data privacy requirements of GDPR, the value and use of machine learning and advanced analytics to drive business initiatives and achieve better business outcomes, and adaption to the accelerating pace of change.  One panel of CDO’s – including Christina Clark of General Electric (GE), Mark Ramsey of Glaxo Smith Kline, and Venkat Varadachary of American Express – addressed a number of these points in detail.

blog MITCDOIQ presentation Notes from 2017 MIT CDOIQ Symposium: CDO’s, and Data, go Mainstream

Managing Data for Business Value and Growth

Though the CDO role emerged out of the financial crisis and the need for broader controls around regulatory requirements and data governance, that role has changed and most new CDO’s are addressing the need for business value and revenue growth.  This doesn’t mean that the foundation and fundamentals of data governance and compliance are forgotten or dropped.  In fact, those aspects are critical to this shift as business growth is dependent on an organization’s ability to structure and manage data for speed and flexibility.  The approaches taken to succeed may vary considerably though.

blog banner landscape Notes from 2017 MIT CDOIQ Symposium: CDO’s, and Data, go Mainstream

One participant noted the example of Google absorbing YouTube.  Google took 2 years to build the infrastructure and information needed to effectively monetize the volumes of data acquired.  In the case of financial services firms, that approach is rare, particularly with mergers.  In those cases, the customer support and client relationship aspects are critical and it is more important to leave the systems in place and develop an approach to span across those systems, even where they have similar data.

The participants particularly noted that the benefits of this data-driven business approach are large, but that there is a need for a clear vision of what the organization wants to do and address.  This includes establishing a scorecard with quantifiable metrics for business value.

Some of these may be as straightforward as: identifying how many different revenue-generating use cases have been deployed; or determining how much faster deployment is with new approaches to data delivery.  Often, it’s the critical data elements, perhaps no more than 50 per subject area, that provide the key metrics around operational value (and the impact to business processes that will break if incorrect) tied in with the costs to acquire, manage, and consume such data.

blog MITCDOIQ trillium Notes from 2017 MIT CDOIQ Symposium: CDO’s, and Data, go Mainstream

I was on hand with colleagues at MITCDOIQ to demonstrate how our Trillium data quality software can help organization’s data governance initiatives 

The CDO of Glaxo Smith Kline noted a goal to change the time to discover new drugs from 8 years to 1 year, and transform the pharmaceutical industry by leveraging sensor-based and genomics data.  Many steps are needed to reach that business goal including standardizing internal data and being able to connect the internal data to new external sources.

Democratization of Data

For individuals in an organization to be effective, they need trusted data at hand to move forward with speed and efficiency.  Data scientists are just a part of that equation.  Sales, marketing, operations, and others in the lines of business all need data.  Getting data into the hands of employees, even if imperfect, is valuable – it creates incentives sooner as people can see the data issues and work to solve them.  Helping solve the problems for these people in accessing and using data not only democratizes the data, but provides them the ability to act in a more agile way with faster time-to-value.

At the same time, it is important to remember that data must be served in a manner that is consumable to these varied users.  Some will want visualizations and dashboards, some need alerts and notifications for faster action, some need data in Excel, and some could care less about visualization and want access via tools such as Python.

To achieve this democratization, it’s important then to understand what data people want access to, how it may be delivered and consumed, and how individuals can accelerate this process.  Shifting the cultural mindset to a process of collecting and accessing data rather than modeling and structuring the data first helps to more readily identify where the business challenges are and how data may be applied to solve the issues and drive value.

Barbara Latulippe, CDO of Dell, reiterated many of the panels themes in her MITCDOIQ presentation on “Governance and Stewardship in the Big Data Era.”  She noted that the data scientists in her organization were struggling to find data.  In one case, it took 35 phone calls by a data scientist to determine all the context around the data!  Democratizing data means it is critical to make the data easy to find, easy to understand, and easy to determine trust and quality.

One metric for Dell is simply reducing the time needed to find and consume data for prescriptive value with a goal to move from 70% of a data scientist’s available time spent in finding data (a statistic regularly reported) to 30% of their time.  Achieving this requires data governance, echoing the earlier panel’s comments that governance is foundational to success in this area.  Dell’s approach follows a Lean Data Governance model, a practice that Trillium Software has noted in the past, including:  starting small, showing success, visualizing results, and breaking down silos by showing others “what’s in it for them.”

Finding Data Skills, Building Data Literacy

On the final day of MITCDOIQ, Natalie Evans Harris, VP of Ecosystem Development at The Impact Lab, discussed the perceived issue in finding individuals with the data skills needed to help organizations achieve business value and growth.  She noted that this is often a “signaling” problem.

The focus by organizations on finding the “data scientist” who can understand and communicate with the business while finding and accessing data, testing hypotheses, building algorithms and models, and ramping these up into ongoing executable frameworks is misguided.  What organizations need to focus on is bringing teams together with the mix of skills that can empower all involved to move the organization forward.  This is the approach noted by Booz, Allen, Hamilton in their Field Guide to Data Science.

It’s important to remember that the range of skills needed to work effectively with data exist in many individuals and consider whether we are really looking for specialists or trying to take advantage of competencies (e.g. biologists, linguists, etc. can provide data science) and blend those with the subject matter experts who understand the business, understand business opportunities, and can present ideas in a manner that makes sense in the organization.

My own topic at MITCDOIQ, Finding Relevance in the Big Data World, touched on an aspect of data literacy, specifically how to approach the challenge of considering what data is important, i.e. relevant, for a given business initiative. Wolf Ruzicka, the Chairman of the Board at EastBanc Technologies, noted in his blog “Grow A Data Tree Out Of The “Big Data” Swamp” of June 1, 2017. “If you don’t know what you want to get out of the data, how can you know what data you need – and what insight you’re looking for?”

A fundamental step then in bringing data into the mainstream is ensuring that the individuals working with the data to establish a goal (whether generating new revenue, meeting compliance goals such as GDPR, or reducing operational costs).  Only with a business goal in mind can you test hypotheses, evaluate and measure data, and determine whether the data is fit for purpose.  The results must be documented in a way that they can be communicated out through a repeatable data governance process.  Such a process should start small, but it provides an approach to build a practice, show success, and build business value while democratizing and measuring the data used and highlighting which data has value for which business purpose.

As Harris noted, it’s important to address change management services and processes, particularly to understand how people can use, interpret, and understand their data and their dashboards.  This means not only thinking about data literacy, but building data literacy!

lessons learned Notes from 2017 MIT CDOIQ Symposium: CDO’s, and Data, go Mainstream

From MITCDOIQ: Lessons Learned in Dell’s CDO/Data Governance Journey

Data-driven Success

As the CDO panel noted, having both data governance and data science teams together in the organization helps ensure that regulatory obligations are met while building for growth.  It’s the underlying foundation needed to achieve success at data-driven initiatives.  And it’s hard to get people bought in fully, and requires culture change, but that is part of the CDO’s work.  This shift is evidenced in even at MITCDOIQ in its topics – no longer is the focus on creating a CDO office but on sharing the stories of organizational change and the adoption of fundamental data-driven processes and data literacy.

Discover the new rules for how data is moved, manipulated, and cleansed – download our new eBook The New Rules for Your Data Landscape today!

Let’s block ads! (Why?)

Syncsort + Trillium Software Blog

5 Big Data Experts Who Caught Your Attention This Year

Not all expert interviews are created equal. Here are the five most popular Big Data experts we’re spoken to this year. Did your favorite make the list?

#1  Dakshinamurthy V. Kolluru

blog kolluru banner 5 Big Data Experts Who Caught Your Attention This Year

As the founder and president of the International School of Engineering, aka INSOFE (@INSOFEedu), Kolluru could teach his own master class in communicating complex ideas with clarity and excitement.

Kolluru has done extensive work in data science, particularly mathematical algorithms and pattern extraction. He has helped establish several data science centers of excellence, and proactively steered INSOFE into the globally acclaimed School of Applied Engineering that it is today.

We asked him about the use of data science skills in the workplace, and where the field of data analytics was headed.

See what he had to say about Data Science and Big Data Analytics

#2  Wayne W. Eckerson

blog eckerson banner 5 Big Data Experts Who Caught Your Attention This Year

Eckerson (@weckerson), the founder and principal consultant for Eckerson Group, is an internationally recognized thought leader in the business intelligence. He is a sought-after consultant, noted speaker and bestselling author.

In our interview, Eckerson offers insight on the evolution of business analytics with additional thoughts on what business intelligence professionals will need to do survive the future of self-service technology.

Read his interview for more on the Evolution of Business Intelligence

#3  Reynold Xin, Databricks

blog xin banner 5 Big Data Experts Who Caught Your Attention This Year

Xin (@rxin) is the chief architect for Spark core at Databricks, and also one of Spark’s founding fathers. At this year’s Strata + Hadoop World in San Jose, he gave a presentation on the full history of Spark, from taking inspiration from mainframe databases to the cutting edge features of Spark 2.x.

Syncsort’s Paige Roberts sat down with Xin to get the details on the driving factors behind Spark 2.x and its newest features, such as structured streaming.

Follow along in his popular two-part series:

#4  Neha Narkhede, Confluent

blog neha banner 5 Big Data Experts Who Caught Your Attention This Year

Narkhede (@nehanarkhede) is the co-founder and chief technology officer at Confluent. Prior to Confluent, she built Apache Kafka with two of her colleagues at LinkedIn.

At a recent Confluent partner event, Syncsort’s Paige Roberts spoke with Narkhede about Apache Kafka, Confluent, the future of streaming data processing, and what it’s like to be a “Girl Boss” in Big Data.

You won’t want to miss any of this three-part conversation:

#5  Robert Corace, SoftServe

blog corace banner 5 Big Data Experts Who Caught Your Attention This Year

Corace is the executive vice president of digital disruption at SoftServe, a digital solutions company. As a seasoned industry professional with more than 25 years of experience leading sales and implementation teams, technology groups and global delivery centers, Corace is an expert in digital and technology enablement, digital transformation, digital disruption and digital futurism.

We checked in with Robert to get his insight on Big Data and current trends and challenges in data and the Internet of Things (IoT).

Check out his two-part interview:

More Big Data Experts

blog banner BBDtL ExpertsSay 5 Big Data Experts Who Caught Your Attention This Year

For additional words of wisdom from Big Data experts, download our free eBook: Bringing Big Data to Life: What the Experts Say

Let’s block ads! (Why?)

Syncsort + Trillium Software Blog

Representing a simple hierarchical list in SQL Server with JSON, YAML, XML and HTML.

How difficult can it be to produce a simple hierarchical list in JSON, YAML, XML and HTML from a SQL Server table that represents a simple hierarchy within an organisation. Well once you know, it is easy and William Brewer is on a mission to tell you how

JSON, XML, YAML and HTML are great for recording hierarchies such as organisations, taxonomies, and parts lists. How do we output structured document fragments to show a hierarchical list using SQL? I was hoping that the advent of JSON to SQL Server would make this easier but I found its use frustrating to the point that I keep it as arms-length as possible.

Because I would have found it useful myself, I’ve recorded here how to use T-SQL to get the four main types of document types to represent a simple hierarchical list in SQL Server.

First, before we do anything else, we’ll create some test data. In this example, I’ll steal the employee hierarchy from AdventureWorks2014, and put it in a test table.

First, we’ll tackle a JSON rendering of the hierarchy

We can now try it out …

To get this JSON rendition (after prettifying to make it easier to read)

So, emboldened, we try YAML. It turns out to be very easy as there is no support for it in SQL Server

This will render the same manager and his reports even more simply, and doesn’t need prettifying to understand

This, when executed gives the following YAML document …

The XML version is pretty simple

Giving the following XML (prettified to make it easier to read) …

And finally, for the sake of completeness here is the HTML List version

Which can be executed like this

…to give this HTML fragment …

So here we have it, all four commonly-used document types used for hierarchical lists, output from SQL Server. These are fairly simple to elaborate, and apologies in advance for any errors.

Let’s block ads! (Why?)

SQL – Simple Talk

Online Analysis Services Course: Developing a Multidimensional Model

Check out the excellent, new online course by Peter Myers and Chris Randall for Microsoft Learning Experiences (LeX). Lean how to develop multidimensional data models with SQL Server 2016 Analysis Services. The complete course is available on edX at no cost to audit, or you can highlight your new knowledge and skills with a Verified Certificate for a small charge. Enrollment is available at edX.

Let’s block ads! (Why?)

Analysis Services Team Blog

Plot the results of a nonlinear system of equations

I have this system of four nonlinear equations:
pzaU7 Plot the results of a nonlinear system of equations

the unknowns are a1, a2, gamma1 and gamma 2. all other parameters are known. What I want is to plot the a1 vs sigma2 as below:
9luQC Plot the results of a nonlinear system of equations

I tried findinstance but it takes a lot of time to find the results for any value of sigma2, Let alone plotting a1 for different values of sigma2!
Can anybody help me?
Thanks in advance

Let’s block ads! (Why?)

Recent Questions – Mathematica Stack Exchange

Mobile Programmatic Ad Fraud Is Big With Entertainment And Education Apps

Many Chinese companies have risen on the back of mobile advertising, but a new report says fraud is rampant in the sector.

Global ad revenue wasted on fraudulent traffic could reach $ 16.4 billion in 2017, according to information from AppLift China. AppLift conducted a twelve-week study and found out that certain app categories are more susceptible to fraud than others, with entertainment apps and news apps seeing the most fraud traffic.

Mobile app categories with the most fraudulent activity include entertainment, news, and education with 22%, 22% and 21%, respectively. Meanwhile the apps with the least fraudulent activity dealt with parenting and shopping with 2% and 3%, respectively. More complex and sophisticated types of fraud have evolved such as click injection and click stuffing.

“As a leader in the mobile ad space, we firmly believe it is our responsibility to not only focus our efforts on detecting and fighting ad fraud, but to also share our learnings with industry peers so that we can all better understand the risks and fight fraud together,” said Tim Koschella, CEO of AppLift.

Let’s block ads! (Why?)

ChinaWirelessNews.com

Xiaomi Dumps LG, Signs Agreement With Samsung

Chinese smartphone maker Xiaomi signed an agreement with Samsung Display for the supply of OLED screens, which will be used on Xiaomi’s flagship smartphones in 2018.

Samsung will supply 6.01-inch rigid OLED panels to Xiaomi. The first batch will be shipped in December 2017. About one million panels will be shipped in the first month and 2.2 million more will be shipped in the second month. Financial terms of the deal were not released.

Xiaomi originally planned to use LG’s 5.49-inch flexible OLED panels. Unfortunately, the two parties did not reach an agreement because of the operation delay of LG’s new factory.

Media reports reveal that LG’s new factory E5 is expected to start operation in August 2017, which is about three months later than its original plan. The reason for the delay is because LG is upgrading its panel resolution from full HD to QHD, with 2156×1440 pixels.

Let’s block ads! (Why?)

ChinaWirelessNews.com

Xiaomi Dumps LG, Signs Agreement With Samsung

Chinese smartphone maker Xiaomi signed an agreement with Samsung Display for the supply of OLED screens, which will be used on Xiaomi’s flagship smartphones in 2018.

Samsung will supply 6.01-inch rigid OLED panels to Xiaomi. The first batch will be shipped in December 2017. About one million panels will be shipped in the first month and 2.2 million more will be shipped in the second month. Financial terms of the deal were not released.

Xiaomi originally planned to use LG’s 5.49-inch flexible OLED panels. Unfortunately, the two parties did not reach an agreement because of the operation delay of LG’s new factory.

Media reports reveal that LG’s new factory E5 is expected to start operation in August 2017, which is about three months later than its original plan. The reason for the delay is because LG is upgrading its panel resolution from full HD to QHD, with 2156×1440 pixels.

Let’s block ads! (Why?)

ChinaWirelessNews.com

Data Governance Review: From Basics to the Latest News & Trends

Similar to last week’s Data Quality Study Guide, we wanted to continue to take advantage of the slower summer season to review the latest in data governance. Take a break from the heat and spend a few moments to get yourself caught up.

New to Data Governance?

If you’re just getting started, we’ve got the perfect data governance primer for you. This article explains the basics of developing an effective data governance process for taming unruly data.

blog data governance policies Data Governance Review: From Basics to the Latest News & Trends

Data Governance + Data Quality = Trust

Data governance requires data quality because ensuring data quality is the only way to be certain that your data governance policies are consistently followed and enforced. It’s likely that is why both data governance and data quality were top of mind at this year’s Collibra Data Citizens event.

At this year’s Data Governance and Information Quality Conference (DGIQ), our own Keith Kohl lead the session about how data governance and data quality are intrinsically linked, and as the strategic importance of data grows in an organization, the intersection of these practices grows in importance, too.

During her Enterprise Data World presentation, Laura Sebastian-Coleman of the Data Quality Center of Excellence Lead for Cigna, noted specifically that data quality depends on fitness for purpose, representational effectiveness and data knowledge. And, without this knowledge, which depends on the data context, our data lakes or even our data warehouses are doomed to become “data graveyards.”

As our new eBook “The New Rules for Your Data Landscape” points out, data is shifting from IT to the business. The result is a new data supply chain which impacts data movement, manipulation and cleansing.

blog banner landscape Data Governance Review: From Basics to the Latest News & Trends

Today’s business leaders rely on Big Data analytics to make informed decisions. But according to figures presented at the recent Gartner Data and Analytics Summit, C-Level executives believe that 33% of their data is inaccurate.

It appears there is an abundance of data, but a scarcity of trust, and the need for data literacy. It’s important to understand what your data MEANS to your organization. Defining data’s value wedge may be key to developing confidence in your enterprise data.

For more information about the data value wedge, watch this educational webcast hosted by ASG and Trillium Software. The recorded discussion explores the importance – and challenge – of determining what data MEANS to your organization, as well as solutions to empower both your technical (IS) and business users (DOES) to collaborate in an efficient, zero-gap-lineage user interface.

Data Governance for Hadoop

Keeping track of data, data security, data access, and regulatory compliance are more critical and more challenging than ever before. Data governance in Hadoop — including auditing, lineage, and metadata management — requires a scalable approach that is easy to interoperate across multiple platforms.

In 2015, Syncsort joined Cloudera to provide a unified foundation for open metadata and end-to-end visibility for governance, effectively bridging the gap between mainframe and Hadoop.

Just last year, Hortonworks CTO Scott Gnau recognized that data governance in Hadoop was still in early development, but definitely a priority at his organization.

At this year’s DataWorks Summit, Gnau made a joint appearance on theCUBE with Syncsort CTO Tendü Yoğurtçu. Gnau was bullish on Hortonworks’ partnership with Syncsort, pointing out that it is built on the foundation of accelerating joint customers time to value and leveraging our mutual strengths.

Syncsort’s Focus on Data Governance

Also during her DataWorks theCUBE appearance, Yoğurtçu explained how the Trillium Software acquisition has been transformative for Syncsort, allowing the organization to deliver joint solutions from data integration and data quality & profiling portfolios. She shared that recent first steps have been focused on data governance use cases leveraging Trillium’s solutions.

Yoğurtçu also touched on the recent announcement of  Syncsort’s partnership with Collibra, noting the importance of making business rules and technical metadata available thru dashboards for data scientists.

For more information on how data governance is changing to match the new flow of data delivery, download our new eBook:The New Rules for Your Data Landscape

Let’s block ads! (Why?)

Syncsort + Trillium Software Blog

ProBeat: Wearables are gimmicks

 ProBeat: Wearables are gimmicks

It’s been a tough month for wearables. Two weeks ago, The Information revealed that Jawbone is being liquidated. This week, CNBC reported that Intel had axed its wearables division.

As my colleagues and friends know, I’m one of the biggest skeptics of wearables in the world. As such, neither of these stories shocked me in the slightest.

At the same time, though, I’m also incredibly bullish on what wearables will one day accomplish. The technology just isn’t here yet.

None of today’s wearables excite me (many concepts and prototypes do, but that’s the case for almost any space). I’ve thought about this for a long time, and the reality is that wearables simply don’t do anything that I wish they could.

I want a device that can truly accomplish what my phone can’t. I don’t care for a wearable that can tell the time, make phone calls, send messages, run apps, and count my steps.

I don’t want a shitty phone on my wrist. Nor on my face.

Google Glass Explorer Edition relaunched this week as Enterprise Glass Edition. I’m happy to see that Google has found a niche for the product, but it’s depressing the company has put the dream of prescription glasses and contact lenses with AR functionality on the back burner.

I want a device that can monitor exactly what I’ve consumed and measure what I have gained (or lost) from it. I want a device that can measure how long I’ve rested and whether it is enough for the life I live. I want a device that can determine what my body really needs based on the information it gathers. That means anything from a recommendation to go for a run today because I’ve been immobile for too long or to eat a specific vegetable because I’m missing a given nutrient.

I strongly believe this is coming. But until the technology arrives, I’m not surprised that startups are folding and tech giants are looking elsewhere.

The good news is that many people do find wearables in their current iteration to be useful. Companies are clearly interested in augmented glasses, while consumers are still buying smartwatches and fitness trackers.

Indeed, IDC estimated last month that wearables grew 17.9 percent in Q1 2017. The top five companies (Xiaomi, Apple, Fitbit, Samsung, and Garmin) aren’t throwing in the towel.

As long as there is at least some demand, money will be invested in the space. And hopefully, those investments will one day pay off with a device that stands on its own.

Attached to your body, of course.

ProBeat is a column in which Emil rants about whatever crosses him that week.

Let’s block ads! (Why?)

Big Data – VentureBeat