• Home
  • About Us
  • Contact Us
  • Privacy Policy
  • Special Offers
Business Intelligence Info
  • Business Intelligence
    • BI News and Info
    • Big Data
    • Mobile and Cloud
    • Self-Service BI
  • CRM
    • CRM News and Info
    • InfusionSoft
    • Microsoft Dynamics CRM
    • NetSuite
    • OnContact
    • Salesforce
    • Workbooks
  • Data Mining
    • Pentaho
    • Sisense
    • Tableau
    • TIBCO Spotfire
  • Data Warehousing
    • DWH News and Info
    • IBM DB2
    • Microsoft SQL Server
    • Oracle
    • Teradata
  • Predictive Analytics
    • FICO
    • KNIME
    • Mathematica
    • Matlab
    • Minitab
    • RapidMiner
    • Revolution
    • SAP
    • SAS/SPSS
  • Humor

Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

December 14, 2018   Self-Service BI

In November, we announced Power BI’s self-service data preparation capabilities with dataflows, making it possible for business analysts and BI professionals to author and manage complex data prep tasks using familiar self-service tools. Dataflow data can be easily shared across Power BI, allowing business analysts and BI professionals to save time and resources by building on each other’s work, instead of duplicating it, leading to more unified, less siloed data. Learn more about Power BI data prep capabilities here.

Today, we’re excited to announce integration between Power BI dataflows and Azure Data Lake Storage Gen2 (preview), empowering organizations to unify data across Power BI and Azure data services. With this integration, business analysts and BI professionals working in Power BI can easily collaborate with data analysts, engineers, and scientists working in Azure. These new features free valuable time and resources previously spent extracting and unifying data from different sources, so your team can focus on turning data into insights.

Data is a company’s most valuable asset. Business analysts and data professionals spend a great deal of time and effort extracting data from different sources and getting semantic information about the data, which is often trapped in the business logic that created it, or stored away from the data, making collaboration harder and time to insights longer.

To address these challenges, Power BI and Azure data services have teamed up to leverage Common Data Model (CDM) folders as the standard to store and describe data, with Azure Data Lake Storage as the shared storage layer. CDM folders contain schematized data and metadata in a standardized format, to facilitate data exchange and to enable full interoperability across services that produce or consume data stored in an organization’s Azure Data Lake Storage account.

One of the compelling features of dataflows is the ease with which any authorized Power BI user can build semantic models on top of their data. Because dataflows already store data in CDM folders, the integration between Power BI and Azure Data Lake makes it possible for any authorized person or service to easily leverage dataflow data, using CDM folders as a shared standard.

Furthermore, with the introduction of the CDM folder standard and developer resources, authorized services and people can not only read, but also create and store CDM folders in their organization’s Azure Data Lake Storage account. Once a CDM folder has been created in an organization’s Data Lake Storage account, it can be added to Power BI as a dataflow, so you can build sematic models on top of the data in Power BI, further enrich it, or process it from other dataflows.

The diagram below showcases a range of services contributing to and leveraging data from CDM folders in a data lake.

20C80CE2 03DF 4982 BB2F C5BE8F02D4A4 Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

Power BI customers can now:

  • Connect an Azure Data Lake Storage Gen2 account to Power BI
  • Configure workspaces to store dataflow definition and data files in CDM folders in Azure Data Lake
  • Attach CDM folders created by other services to Power BI as dataflows
  • Create datasets, reports, dashboards, and apps using dataflows created from CDM folders in Azure Data Lake

These new Power BI capabilities are available today for Power BI Pro, Power BI Premium and Power BI Embedded customers. All you need to get started is an Azure Data Storage account.

Store dataflow data in your organization’s Azure Data Lake Storage

Before you can start storing Power BI dataflows in your organization’s Azure Data Lake Storage account, your administrator needs to connect an Azure Data Lake Storage account to Power BI. Once connected, Power BI administrators can allow Power BI users to configure their workspaces to use the Azure storage account for dataflow storage.

58F9302A 6887 470B 92D2 D41EDF19CF92 Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

Assign workspaces to your Azure Data Lake Storage Gen2

Once a dataflow storage account has been configured for Power BI and storage assignment permissions have been enabled, workspace admins can configure dataflow storage setting. By default, dataflow definition and data files will be stored in Power BI provided storage. Turn on dataflow storage for your workspace to store dataflows in your organization’s Azure Data Lake Storage:

C2BDD4BE 6E35 48ED 8BE6 F12FD32EDD29 Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

Once saved, dataflows created in the workspace will store their definition files and data in your organizations Azure Data Lake Storage account.

Read data from CDM folders created by other services with Power BI

Power BI is only one of the services that can create CDM folders. Azure data services and developer resources can also be used to create and store CDM folders in Azure Data Lake Storage. Once in Data Lake Storage, CDM folders can be easily added to Power BI and used as dataflows—you can use Power BI Desktop and the Power BI service to create datasets, reports, dashboards, and apps using data from the CDM folder, just as you would with a dataflow authored in Power BI. To make this process as simple as possible, we added a new option when creating a new a dataflow in Power BI, allowing you to attach an external CDM folder to a new dataflow:

4A7C6281 6654 4618 98C7 9649EA486FC2 Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

Adding a CDM folder to Power BI is easy, just provide a name and description for the dataflow and the location of the CDM folder in your Azure Data Lake Storage account:

181F55D1 EB76 4FB3 AF2A 4C0F887F1E2D Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

And that’s it. You can now leverage the Power BI dataflow connector to view the data and schema exactly as you would for any dataflow.

You can get started with tutorials and samples and learn how data sharing between Power BI and Azure data services using CDM folders can break down data silos and unlock new insights in your organization. In this tutorial, Power BI dataflows are used to ingest key analytics data from the Wide World Importers operational database into the organization’s Azure Data Lake Storage account. Then, Azure Databricks is used to format and prepare data and store it in a new CDM folder in Azure Data Lake.  Azure Machine Learning reads data from the CDM folder to train and publish a machine learning model that can be accessed from Power BI, or other applications, to make real-time predictions. In parallel, the data from the CDM folder is loaded into staging tables in an Azure SQL Data Warehouse by Azure Data Factory, where it’s transformed into a dimensional model.

The diagram below illustrates the samples scenario showing how services can interoperate over Azure Data Lake with CDM folders:

F62DB348 4582 4DFB 8E15 48A7C1794E99 Power BI Dataflows and Azure Data Lake Storage Gen2 Integration Preview

Today, Power BI and Azure data services are taking the first steps to enable data exchange and interoperability through the Common Data Model and Azure Data Lake Storage. We are continuously working to add new features. Please visit the Power BI community and share what you’re doing, ask questions, or submit new ideas.

Let’s block ads! (Why?)

Microsoft Power BI Blog | Microsoft Power BI

Azure, data, dataflows, Gen2, Integration, lake, Power, Preview, Storage
  • Recent Posts

    • Dapper Duo
    • AI Weekly: These researchers are improving AI’s ability to understand different accents
    • Why Choose RapidMiner for Your Data Science & Machine Learning Software?
    • How to Use CRM Integration to Your Advantage – Real World Examples
    • WATCH: ‘Coming 2 America’ Movie Review Available On Amazon Prime & Amazon
  • Categories

  • Archives

    • March 2021
    • February 2021
    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
© 2021 Business Intelligence Info
Power BI Training | G Com Solutions Limited