Tag Archives: data

Data privacy startup Mine raises $9.5 million and expands to the U.S.

October 21, 2020 Big Data

Mine, which is developing a platform that helps discover and manage personal data on the web, today closed a $ 9.5 million funding round led by Gradient Ventures, Google’s AI-focused venture firm. The cash infusion comes as Mine prepares to launch in the U.S. following debuts in Israel and Europe earlier in the year. According to a spokesperson, 100,000 people have used Mine’s app to send over 1.3 million data-reclaim requests to more than 150,000 companies to date.

A majority of people in the U.S. (60%) believe their online and offline activities are being tracked and monitored by companies and the government with some regularity, according to a Pew Research Center survey. It’s a sentiment reflected in reports like PricewaterhouseCoopers’ Consumer Intelligence Series, which found that only 10% of consumers felt they had complete control over their personal information.

Mine’s technology discovers companies that are holding data by analyzing email subject lines, as opposed to the actual content of emails (e.g., “Welcome to Facebook” or “Your Amazon Receipt”). Once connected with an inbox, Mine maps all companies a user has interacted with via email, generating visualizations that show the type of data collected and the corresponding risk level. A dashboard provides a detailed overview of this data, along with shortcuts to remove the data from services no longer in use.

Mine’s algorithms identify companies most likely to have collected personal data based on the sender and the number of times they appear in an inbox. (According to Mine, almost 90% of traces can be found through email subject lines.) The platform independently locates and scans these companies’ privacy policies and data structures to determine what kinds of information they’re collecting from users.

An initial scan takes about 30 seconds, after which the system continues in the background. Mine sends a notification and an email once the audit is completed, and the app continuously provides updates about new traces of companies in a footprint. Mine claims users typically have a 40% cleaner footprint by the end of a session. The company also claims 15,000 Mine users have narrowly avoided data breaches since 2018 by deleting their data right before the breach occurred.

Mine, which has offices in Tel Aviv and New York, is free to use in 10 languages for Gmail, Yahoo, and Microsoft Outlook accounts, though the company plans to eventually charge a subscription fee. For now, it will focus on products that enable businesses to automatically process “right-to-be-forgotten” requests, helping people demand the deletion of certain data pertaining to them. In May 2018, the European Union’s General Data Protection Regulation (GDPR) codified right-to-be-forgotten protections into law. This was followed in January 2020 by the California Consumer Privacy Act (CCPA), which similarly requires that people be able to access and delete certain data.

“Having been part of the tech world from different angles, I noticed how people, myself included, started to lose trust in how companies are collecting and treating personal data. My cofounders and I created Mine to do what we know best — use technology as a solution to shape a new future of equal choice and control to the internet,” CEO Gal Ringel told VenureBeat via email. “Our smart non-intrusive AI makes it easy for people to take back ownership of their online data and minimize digital risks without changing their online behavior.”

Mine’s Gradient Ventures-led round included participation from E.ventures, MassMutual Ventures, and existing investors Battery Ventures and Saban Ventures. It brings the company’s total raised to over $ 12.5 million.

Let’s block ads! (Why?)

Big Data – VentureBeat

Data Analytics Customers Value Choice and Simplicity; Teradata’s New Flexible Cloud Pricing Provides Both

October 21, 2020 BI News and Info

teradata pricing estimate meet business needs Data Analytics Customers Value Choice and Simplicity; Teradata’s New Flexible Cloud Pricing Provides Both

Innovative cloud pricing models include Blended Pricing – for the lowest cost at scale – and Consumption Pricing – for a true pay-as-you-go, usage-based offer

Recognizing that data analytics workloads, usage patterns, and utilization rates can vary widely across an organization, Teradata (NYSE: TDC), the cloud data analytics platform company, today announced flexible cloud pricing options to make it easy for enterprises to grow, and benefit from data analytics in the cloud.

In keeping with Teradata’s aim to provide its customers with simplicity and choice, the company now offers two flexible cloud pricing models: Blended and Consumption. Blended Pricing is best suited for high usage and provides the ultimate in billing predictability while delivering the lowest cost at scale. Consumption Pricing is an affordable, pay-as-you-go option best suited for ad hoc queries and workloads with typical or unknown usage that delivers cost transparency for easy departmental chargeback.

With broad availability of both models, enterprises can expect more choice, lower risk, higher efficiency, and greater transparency from Teradata. These options are crucial in today’s unpredictable market where technologies, supply chains, and customer expectations can shift abruptly, leaving companies with stranded data analytics investments if their software fails to provide enough flexibility to evolve as needs change.

“If 2020 has taught us anything, it’s that change happens fast, and having simple, flexible cloud pricing options gives customers the freedom needed to optimize their data analytics investments,” said Hillary Ashton, Chief Product Officer at Teradata. “Different analytic use cases have vastly different utilization patterns at different points in time, which means that having choice in pricing models enables Teradata to offer the best one for each customer scenario ranging from a small ad hoc discovery system to a large production analytics environment.”

Through decades of catering to data analytics needs spanning dozens of business cycles, Teradata understands deeply what is required for firms to extract the most value from their data. Blended Pricing, which is based primarily on capacity, is optimized for high or predictable utilization. Consumption Pricing is optimized for typical to low utilization, unknown future usage, or for tactical business analytics that include frequent ad hoc queries. The ability for Teradata customers to choose the most appropriate cloud pricing model for data analytic workloads facilitates financial governance and can improve customers’ return on investment.

“The convenience of a true consumption-based pricing model – determined by actual usage of the Vantage platform for running successful queries, rather than just available capacity – is a win for customers who want to better align their investment with specific analytic outcomes,” added Ashton. “It also corrects outdated perceptions about the cost required to become a Teradata customer, since there is now a risk-free, zero down option to pay only for what’s used with Vantage, the industry’s best cloud data analytics platform.”

Analyst firm Gartner predicts in the report, Overcome Economic Uncertainty Through Financial Governance of Your Cloud Data Management Environment, April 2020 that, “By 2022, 75% of organizations using cloud data management will encounter budget overruns resulting in their questioning the value of using cloud services.”

In order to ensure the benefits of deploying in an agile cloud environment are not negated by lack of pricing transparency and cost overruns, Teradata has developed flexible cloud pricing options to meet every company’s use case(s):

Blended Pricing

Teradata’s Blended Pricing option is an enterprise-class pricing model that couples low-cost, predictable reserved compute capacity with on-demand elasticity. Customers pay only for compute resources used or activated, plus storage.

Blended Pricing allows organizations to get cloud elasticity when needed while also benefitting from the lowest cost at scale.

Key benefits of Teradata’s Blended Pricing model include:

Predictability: No billing surprises – unlike other cloud analytics companies that employ inefficient auto-scaling which drives up incremental cost;
Lowest cost at scale: Vantage scales linearly, whereas other cloud analytics companies get increasingly inefficient as they scale; and
Granularity: Vantage offers incremental elasticity, which means not over-provisioning, and thus not over-paying for what’s not needed, unlike alternatives.

Consumption Pricing

Teradata’s Consumption Pricing is an innovative, usage-based option with automatic elasticity in which organizations pay only for compute resources consumed for successful queries, measured down to the kilobyte by user, plus storage. Businesses leveraging Teradata’s Consumption Pricing model need never worry about utilization, system sizing, or resource status since Teradata manages them on their behalf – and they never pay for what they do not use. For a risk-free experience, organizations can even start with zero down and no obligation.

Consumption Pricing makes it easier for customers to experiment with new analytics without fear of leaving money for unused resources on the table. Longtime Teradata customer, True Value Company – one of the world’s leading hardlines wholesalers – is one such customer now taking advantage of the company’s new Consumption Pricing model.

“As we continue on our modernization journey, it’s imperative that we are able to drive new, user-driven analytics capabilities to uncover critical business insights and operational efficiencies, all at an optimized cost,” said Ron Byczynski, Director, Infrastructure, DevOps & QA at True Value. “By leveraging Teradata’s Consumption Pricing model, we will pay only for resources used, allowing us to better streamline our IT investments to ensure maximum ROI.”

Key benefits of Teradata’s Consumption Pricing model include:

Pay only for what is used: Enables improved business efficiency and aligns investment with outcomes while not paying for unused capacity;
Automated elasticity: Since resource sizing is embedded in the offer, now customers no longer need to think about sizing, saving them time on capacity planning;
Cost allocation: Easy departmental chargeback on data analytics costs based on users’ actual resource consumption; and
Zero obligation: A risk-free opportunity to use Teradata Vantage in the cloud.

Vantage: the modern cloud data analytics platform

Teradata Vantage is the modern cloud data analytics platform that unifies data lakes, data warehouses, and analytics with new data sources and types. Built for a hybrid multi-cloud reality, Vantage solves the world’s most complex data challenges at scale.

Both Teradata’s Blended and Consumption Pricing models offer customers the best of what the Vantage platform has to offer, including:

As-a-service delivery: Teradata provisions and manages each customer in a dedicated public cloud environment, enabling users to focus on answers instead of non-differentiating tasks;
Dynamic resource allocation: Field-proven workload management helps customers prioritize users and queries, set guaranteed performance levels, and control spend; and
Software optimization: Sophisticated algorithms minimize compute and IO consumption, streamlining throughput and often eliminating the need to add the non-budgeted compute resources required by others.

Availability

Blended Pricing is now available for every Vantage deployment option worldwide.

Consumption Pricing is available for Vantage on AWS and Vantage on Azure globally. Availability for Vantage on Google Cloud is expected within the first half of 2021.

Resources

Gartner, Overcome Economic Uncertainty Through Financial Governance of Your Cloud Data Management Environment, April 2020.

Note to Investors

This release contains forward-looking statements within the meaning of Section 21E of the Securities and Exchange Act of 1934. Forward-looking statements generally relate to opinions, beliefs and projections of expected future financial and operating performance, business trends, and market conditions, among other things. These forward-looking statements are based upon current expectations and assumptions and involve risks and uncertainties that could cause actual results to differ materially, including the factors discussed in this release and those relating to the timely and successful development, production or acquisition, availability and/or market acceptance of new or existing products, product features and services, and other factors described from time to time in Teradata’s filings with the U.S. Securities and Exchange Commission, including its annual report on Form 10-K for the year ended December 31, 2019 and subsequent quarterly reports on Forms 10-Q, as well as the Company’s annual report to stockholders. Teradata does not undertake any obligation to publicly update or revise any forward-looking statements, whether as a result of new information, future events or otherwise, except as required by law.

Let’s block ads! (Why?)

Teradata United States

Recognizing data points that signal trends for the future of business post-pandemic

October 18, 2020 Big Data

Planning for a post-COVID-19 future and creating a robust enterprise strategy require both strategic scenario planning and the ability to recognize what scenario planners call “news from the future” — data points that tell you whether the world is trending in the direction of one or another of your imagined scenarios. As with any scatter plot, data points are all over the map, but when you gather enough of them, you can start to see the trend line emerge.

Because there are often many factors pushing or pulling in different directions, it’s useful to think of trends as vectors — quantities that are described by both a magnitude and a direction, which may cancel, amplify, or redirect each other. New data points can also show whether vectors are accelerating or decelerating. As you see how trend vectors affect each other, or that new ones need to be added, you can continually update your scenarios.

Sometimes a trend itself is obvious. Twitter, Facebook, Google, and Microsoft each announced a commitment to new work-from-home policies even after the pandemic. But how widespread will this be? To see if other companies are following in their footsteps, look for job listings from companies in your industry that target new metro areas or ignore location entirely. Drops in the price or occupancy rate of commercial real estate, and how that spills over into residential real estate, might add or subtract from the vector.

Think through possible follow-on effects to whatever trend you’re watching. What are the second-order consequences of a broader embrace of the work-from-home experience? Your scenarios might include the possible emptying out of dense cities that are dependent on public transportation and movement from megacities to suburbs or to smaller cities. Depending on who your workers and your customers are, these changes could have an enormous impact on your business.

What are some vectors you might want to watch? And what are examples of news from the future along those trend lines?

The progress of the pandemic itself. Are cases and deaths increasing or declining? If you’re in the U.S., Covid Act Now is a great site for tracking the pandemic. This suggests that pandemic response won’t be a “one and done” strategy, but more like what Tomas Pueyo described in his essay “The Hammer and the Dance,” in which countries drop the hammer to reduce cases, reopen their economies, see recurrences, and drop the hammer again, with the response increasingly fine-grained and local as better data becomes available. As states and countries reopen, there is a lot of new data that will shape all of our estimates of the future, albeit with new uncertainty about a possible resurgence (even if the results are positive).

Is there progress toward treatment or a vaccine? Several vaccine candidates are in trials, and new treatments seem to improve the prognosis for the disease. A vector pushing in the other direction is the discovery of previously missed symptoms or transmission factors. Another is the politicization of public health, which began with masks but may also extend to vaccine denial. We may be living with uncertainty for a long time to come; any strategy involving a “return to normal” needs to be held very loosely.

How do people respond if and when the pandemic abates? Whatever comes back is likely to be irretrievably changed. As Ben Evans said, sometimes the writing is on the wall, but we don’t read it. It was the end of the road for BlackBerry the moment the iPhone was introduced; it just took four years for the story to play out. Sometimes a seemingly unrelated shock accelerates a long overdue collapse. For example, ecommerce has been growing its share for years, but this may be the moment when the balance tips and much in-person retail never comes back. As Evans put it, a bunch of industries look like candidates to endure a decade of inevitability in a week’s time.

Will people continue to walk and ride bikes, bake bread at home, and grow their own vegetables? (This may vary from country to country. People in Europe still treasure their garden allotments 70 years after the end of World War II, but U.S. victory gardens were a passing thing.) Will businesses have the confidence to hire again? Will consumers have the confidence to spend again? What percentage of businesses that shut down will reopen? Are people being rehired and unemployment rates going down? The so-called Y-shaped recovery, in which upper-income jobs have recovered while lower-income jobs are still stagnant, has been so unprecedented that it hasn’t yet made Wikipedia’s list of recession shapes.

Are there meaningful policy innovations that are catching on? Researchers in Israel have proposed a model for business reopening in which people work four-day shifts followed by ten days off in lockdown. Their calculations suggest that this would lower transmissibility of the virus almost as well as full lockdown policies, but allow people in many more occupations to get back to work, and many more businesses to reopen. Might experiments like this lead to permanent changes in work or schooling schedules? What about other long-discussed changes like universal basic income or a shorter work week? How will governments pay for the cost of the crisis, and what will the economic consequences be? There are those, like Ray Dalio, who think that printing money to pay for the crisis actually solves a long-standing debt crisis that was about to crash down on us in any case. Others disagree.

Are business models sustainable under new conditions? Many businesses, such as airlines, hotels, on-demand transportation, and restaurants, are geared very tightly to full occupancy. If airlines have to run planes with half as many passengers, will flights ever be cheap enough to attract the level of passengers we had before the pandemic? Could “on demand” transportation go away forever? Uber and Lyft were already unprofitable because they were subsidizing low prices for passengers. Or might these companies be replaced as the model evolves, much as AOL yielded online leadership to Yahoo!, which lost it in turn to Google? (My bet is that algorithmic, on-demand business models are still in their infancy.)

These topics are all over the news. You can’t escape them, but you can form your own assessment of the deeper story behind them and its relevance to your strategy. Remember to think of the stories as clustering along lines with magnitude and direction. Do they start to show patterns? More importantly, find vectors specific to your business. These may call for deep changes to your strategy.

Also remember that contrarian investments can bring outsized returns. It may be that there are markets that you believe in, where you think you can make a positive difference for your customers despite their struggles, and go long. For O’Reilly, this has been true of many technologies where we placed early bets against what seemed overwhelming odds of success. Chasing what’s “hot” puts you in the midst of ferocious competition. Thinking deeply about who needs you and your products and how you can truly help your customers is the basis for a far more robust strategy.

The audio problem:

Learn how new cloud-based API solutions are solving imperfect, frustrating audio in video conferences. Access here

Let’s block ads! (Why?)

Big Data – VentureBeat

Quantitative and Qualitative Data: A Vital Combination

October 18, 2020 Sisense

We live in a world of data: There’s more of it than ever before, in a ceaselessly expanding array of forms and locations. Dealing with Data is your window into the ways organizations and data teams tackle the challenges of this new world to help their companies and their customers thrive.

Almost every modern organization is now a data-generating machine. As soon as a company’s systems are computerized, the data-generation engine starts up. When these systems connect with external groups — customers, subscribers, shareholders, stakeholders — even more data is generated, collected, and exchanged. And, as industrial, business, domestic, and personal Internet of Things devices become increasingly intelligent, they communicate with each other and share data to help calibrate performance and maximize efficiency. The result, as Sisense CEO Amir Orad wrote, is that every company is now a data company.

Smart use of your data can be the key to optimizing processes, identifying new opportunities, and gaining or keeping a competitive edge. But are you paying attention to all of your data? Do you have the means to handle every kind of data? And can you take full advantage of the insights it can reveal? Your answers will depend on whether you can gather and analyze both quantitative and qualitative data. Let’s consider the differences between the two, and why they’re both important to the success of data-driven organizations.

Quantitative and Qualitative Data: A Vital Combination

Digging into quantitative data

Most commonly, we think of data as numbers that show information such as sales figures, marketing data, payroll totals, financial statistics, and other data that can be counted and measured objectively.

This is quantitative data. It’s “hard,” structured data that answers questions such as “how many?” or “how often?” It’s in an organized format, usually rows and columns, and is stored as a relational database, in which there are relationships among these rows and columns.

All descriptive statistics can be calculated using quantitative data. It’s analyzed through numerical comparisons and statistical inferences and is reported through statistical analyses.

As quantitative data is always numeric, it’s relatively straightforward to put it in order, manage it, analyze it, visualize it, and do calculations with it. Spreadsheet software like Excel, Google Sheets, or traditional database management systems all mainly deal with quantitative data. These programs and systems are great at generating basic visualizations like graphs and charts from static data. The challenge comes when the data becomes huge and fast-changing.

Why is quantitative data important?

Quantitative data is often viewed as the bedrock of your business intelligence and analytics program because it can reveal valuable insights for your organization. These numbers show performance, efficiency, reach, market share, revenue in, and expenses out. This is fundamental information when it comes to understanding how your organization is doing.

Additionally, quantitative data forms the basis on which you can confidently infer, estimate, and project future performance, using techniques such as regression analysis, hypothesis testing, and Monte Carlo simulations.

These techniques allow you to:

See trends and relationships among factors so you can identify operational areas that can be optimized
Compare your data against hypotheses and assumptions to show how decisions might affect your organization
Anticipate risk and uncertainty via mathematically modeling

Consequently, using quantitative data, you can make strategic and tactical decisions that will benefit your organization and drive growth.

What are the problems with quantitative data?

Despite its many uses, quantitative data presents two main challenges for a data-driven organization.

First, data isn’t created in a uniform, consistent format. It’s generated by a host of sources in different ways. To access it effectively, you need to organize it and clean it to avoid mistakes, oversights, and repetitions that could compromise the integrity of your data and, by extension, the accuracy of your insights.

The solution to this problem is to ensure that your BI and analytics platform can handle as many sources and forms of data as possible, both on-premises and on the cloud, so that you don’t neglect or miss any of it.

Second, and more challenging, is the fact that not all the information you generate and collect is structured, quantitative data. As data is produced from a growing array of sources, quantitative data may be as little as 20% of all the data available to most organizations. If you focus your efforts entirely on quantitative data, you’ll overlook a huge amount of valuable information, and your insights and decision-making could become distorted as a result. You need a solution that can access and analyze the other 80%: qualitative data.

Exploring qualitative data

Qualitative data is unstructured, meaning it’s not in a predefined format. In its raw state, it comes in a variety of forms: text, social media comments, phone call transcripts, various logs files, images, audio, video, and more.

Often, the information in qualitative data is a categorical variable. This means it describes the characteristics or qualities of data units, such as “what type,” “which category,” “who” (or “which persona”). These create a picture of the context and environment in which the data can help build an understanding of feelings, opinions, and intentions. That’s because qualitative data is concerned with understanding the perspective of customers, users, or stakeholders. This type of data is often collected through less rigid, measurable means than quantitative data. Examples include comments, recordings, interviews, focus group reports, and more, typically received in the natural language of the informants.

Qualitative data is much more subjective, “soft” information than quantitative data. However, advanced analytics can now identify and classify this information and transform it into findings that lead to game-changing insights for organizations.

As the sources of data continue to proliferate, an increasing proportion of it is unstructured and qualitative. It’s more complex, and it requires more storage. Traditional methods of gathering and organizing data can’t organize, filter, and analyze this kind of data effectively. Advanced technology and new approaches are needed. What seem at first to be very random, disparate forms of qualitative data require the capacity of data warehouses, data lakes, and NoSQL databases to store and manage them.

Qualitative data benefits: Unlocking understanding

Qualitative data can go where quantitative data can’t. For example, it’s the gateway to sentiment analysis — understanding how users, customers, and stakeholders think and feel, as well as what they do. Techniques that focus on qualitative data, such as content analysis and narrative analysis, enable you to:

Analyze text to find the most common themes in open-ended data such as user feedback, interview transcripts, and surveys, so you can pinpoint the most important focus areas
Better interpret feelings or perceptions about your organization, product, services, processes, or brand, to help identify what changes or innovations would be most effective

Having access to this information gives you a new dimension of insights on which to base decisions and determine the tactical and strategic direction of your organization. With qualitative data, you can understand intention as well as behavior, thereby making predictive analytics more accurate and giving you fuller insights. You can analyze and learn from the large volume of unstructured data to ensure that your data-driven decisions are as solid as possible.

For example, Skullcandy, the manufacturer of headphones, earbuds, and other audio and wireless products, uses predictive and sentiment analysis to understand customers better. This enables the company’s product development team to explore opportunities for reducing returns and warranty requests for new products before they’re even released and get reactions from customers about current products. Skullcandy uses these insights to hone its new products based on what customers say they like and dislike, so that it can offer more attractive products to its target market.

Getting the most from qualitative data

Qualitative data eludes the rigid categorization and the storage limitations of traditional databases. It provides the raw material for information that is more varied and harder to organize than structured, qualitative data. Making sense of and deriving patterns from it calls for newer, more advanced technology.

Natural language processing (NLP), involving machine learning, is how your BI and analytics platform can understand the meaning of unstructured data such as emails, comments, feedback, and instant messages. It enables search-driven analytics that allow you to process complex NLP, because you can ask the system questions using natural, everyday language. Plus, you can receive visualizations and embed insights into external apps, even if what you’re asking for is relatively complex from an analytical point of view.

Furthermore, systems powered by AI and augmented analytics continuously learn what people choose to do with data. Algorithms identify patterns in the data, and these make search results faster, more accurate, and more complete. You can integrate AI analytics tools with other interfaces like Amazon Alexa and chatbots, drawing on these technologies’ impressive developments in NLP, too.

With all this potential to find new insights, it’s no surprise that 97% of business leaders say their businesses are investing in big data and AI initiatives.

Better together: Working with qualitative data and quantitative data

Quantitative data is the bedrock of your BI and analytics. It provides a solid foundation on which you can build data visualizations and insights that are fundamental to your organization.

With that foundation in place, qualitative data enables you to take your analytics further, get more understanding from a fast-growing array of sources, and power your decision-making with data from even more perspectives.

You have to be able to work with both kinds of data in order to unlock the most comprehensive insights. Together, they can help set the stage for game-changing pivots and innovations that can keep your organization at the head of the pack.

overcoming challenges in nlp Blog Banner 770x250 1 Quantitative and Qualitative Data: A Vital Combination

Adam Murray began his career in corporate communications and public relations in London and New York before moving to Tel Aviv. He’s spent the last ten years working with tech companies like Amdocs, Gilat Satellite Systems, and Allot Communications. He holds a Ph.D. in English literature. When he’s not spending time with his wife and son, he’s preoccupied with his beloved football team, Tottenham Hotspur.

Let’s block ads! (Why?)

Blog – Sisense

How to fit 3 data sets to a model of 3 differential equations?

October 15, 2020 BI News and Info

Let’s block ads! (Why?)

Recent Questions – Mathematica Stack Exchange

DroneDeploy moves beyond drones to let industries analyze visual data captured at ground level

October 14, 2020 Big Data

Since its inception in 2013, DroneDeploy has served to help companies access commercial drone operators and create professional-grade imagery of the built environment from the skies above. But at its annual DroneDeploy Conference today, the San Francisco-based company announced that it’s coming back down to Earth with a new service that captures visuals at ground level.

Thus far, DroneDeploy has enabled companies across industries such as agriculture, construction, insurance, mining, and oil and gas to manage drone fleets and analyze aerial infrastructure data. Now, with 360 Walkthrough as part of its product lineup, DroneDeploy can give companies a more holistic view of their properties through combining data from just about any angle.

The launch comes as numerous companies have emerged to capitalize on the construction industry’s much-maligned inefficiency problem. In particular, startups such as Disperse and OpenSpace recently raised sizable funding rounds to automate photo documentation on construction sites. Expanding its scope to on-the-ground documentation makes a great deal of sense for DroneDeploy, as it means existing clients can use a single company for all their documentation needs.

Automation

As part of its testing phase, DroneDeploy teamed up with construction giant Brasfield & Gorrie, which has been using DroneDeploy to track progress on building sites since 2015. Together, they enlisted the help of Boston Dynamics, the robotics company famed for its viral videos that demonstrate robots doing all manner of crazy things, to develop the workflow required to capture 360-degree video construction documentation from the ground.

DroneDeploy retrofitted Spot, Boston Dynamics’ $ 74,500 robot, with a 360-degree camera to autonomously capture data from inside buildings and snap close-up photos for anyone to inspect remotely — this is particularly notable at the moment, given that social-distancing guidelines may limit the number of people who can safely be present at a build at any given time.

DroneDeploy moves beyond drones to let industries analyze visual data captured at ground level

Above: Boston Dynamics’ Spot uses DroneDeploy to look around a construction site.

Spot can travel pretty much anywhere on a building site, with stairs posing little problem to the canine-esque quadruped.

Above: Spot can climb stairs to document any location on a building site.

Anyone perusing the footage can identify and flag issues on-site, such as a crack that has appeared in a newly built wall or some rogue debris that’s lying around.

Above: DroneDeploy’s software can use captured footage to report issues on-site.

Moreover, DroneDeploy can help site managers compare and contrast footage from different dates to show what changes have happened over a certain time period.

Above: DroneDeploy’s software can compare and contrast footage from different dates.

Although the construction industry served as a core test bed for DroneDeploy’s new on-the-ground photo documentation offering, its true scope extends into just about any sector.

“DroneDeploy’s plan is to continue to innovate with companies that can help automate job sites not just in construction, but agriculture, energy, utilities, and more,” DroneDeploy CTO and cofounder Nick Pilkington told VentureBeat.

Moreover, although Boston Dynamics played a pivotal role in the pilot phase, this was really just to iterate the DroneDeploy service ahead of its proper launch — DroneDeploy will work with most 360-degree cameras, regardless of how they are controlled and operated.

“The cameras used with 360 Walkthrough can be operated manually or automatically,” Pilkington added. “It can be as high-tech as a Boston Dynamics Spot, or as simple as a camera on a stick that a human walks through the site. In other words, any 360 camera mounted on a robot, drone, car, helmet, or even on your phone can be used with 360 Walkthrough.”

Today’s beta launch comes less than a year after DroneDeploy raised $ 35 million in funding, taking its total money raised to nearly $ 100 million. Other players in the space include PrecisionHawk, which also continues to raise sizable funding rounds.

DroneDeploy’s 360 Walkthrough is open to all DroneDeploy customers from today, and it will be made generally available to all enterprises by the end of the year.

Let’s block ads! (Why?)

Big Data – VentureBeat

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

October 13, 2020 Self-Service BI

This weekend I had the pleasure of presenting at SQL Saturday Copenhagen #963 with a session with the title “Power BI Paginated Reports – How to and why”.

I was asked if it was possible to create datadriven report subscriptions in paginated reports and I said “Not yet” – it turns out that is not exactly true.

With the new action “Export to File for Paginated Reports (preview)” – in Power Automate – link

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

It is possible !!!

And yes, for now we need the reports to be stored in a workspace backed by premium capacity but soon it will be available here

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

In my workspace I have a simple report with 2 parameters – Year and Country

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

In order to get a report delivered to multiple recipients with different parameters I created an Excel Workbook in Onedrive for business with 3 different reports I want sent all with different parameters for County and Year.

You can store the data in other sources that you can connect Power Automate with – but for the example I just used Excel file.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Next is to design the flow in Power Automate

In the example I choose the trigger “Manually trigger a flow” – but you can of course choose other triggers to activate this flow – for instance a Recurrence so you can execute the reports at a specified frequency.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Next step is to get the rows from the Excel file – So I use the List rows present in a table

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

This action will return the three rows and next is to use the Control “Apply to each” to loop through each row

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

For the output we choose the value from the “List Row…” step

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Then we add the action “Export To File for Paginated Reports

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

This action will run a selected report from a specific workspace

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

We can choose between the different file formats that is possible for paginated reports – in this example I chose PDF.

Add the end of the Action window – we can specify the values for each of the parameters.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

OBS – You have to type in the Parameter name manually and be aware that the name of the parameter doesn’t necessarily match the label shown

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

As in this case – the Year parameter is called “DateCalendarYear” – and that is the name you must use for “ParametersValues name”

We bind the value of the parameter to the value from the Excel file using the Dynamic content selector.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Next step is to add a step to send the report and bind the fields for the To field – add a subject and in the attachments section we give the attachment a name and then use the result of the “Export to File for paginated reports” as the attachment content

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

The flow is now ready for testing – Save you flow and hit the test button.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Accept the used connections and hit Run Flow and we can go to the flow run overview and follow the steps.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

Hopefully you will see that all steps are successfully.

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

And we can check our mail box to see the report delivered

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

The recipients of the e-mail can be sent to any e-mail address.

Can we pay by the hour ?

In my case I used an Power BI Embedded A4 SKU as capacity – this can be turned on and off so you can expand the flow with actions that can start the capacity and run the subscriptions and then turn the capacity off again.

You only pay for the time the capacity is on – and the list price per hour is $ 8.

Hope this can help you and really looking forward to all exciting stuff that is coming around Power BI paginated reports.

Let’s block ads! (Why?)

Erik Svensen – Blog about Power BI, Power Apps, Power Query

The secrets of small data: How machine learning finally reached the enterprise

October 9, 2020 Big Data

Over the past decade, “big data” has become Silicon Valley’s biggest buzzword. When they’re trained on mind-numbingly large data sets, machine learning (ML) models can develop a deep understanding of a given domain, leading to breakthroughs for top tech companies. Google, for instance, fine-tunes its ranking algorithms by tracking and analyzing more than one trillion search queries each year. It turns out that the Solomonic power to answer all questions from all comers can be brute-forced with sufficient data.

But there’s a catch: Most companies are limited to “small” data; in many cases, they possess only a few dozen examples of the processes they want to automate using ML. If you’re trying to build a robust ML system for enterprise customers, you have to develop new techniques to overcome that dearth of data.

Two techniques in particular — transfer learning and collective learning — have proven critical in transforming small data into big data, allowing average-sized companies to benefit from ML use cases that were once reserved only for Big Tech. And because just 15% of companies have deployed AI or ML already, there is a massive opportunity for these techniques to transform the business world.

The secrets of small data: How machine learning finally reached the enterprise

Above: Using the data from just one company, even modern machine learning models are only about 30% accurate. But thanks to collective learning and transfer learning, Moveworks can determine the intent of employees’ IT support requests with over 90% precision.

Image Credit: Moveworks

From DIY to open source

Of course, data isn’t the only prerequisite for a world-class machine learning model — there’s also the small matter of building that model in the first place. Given the short supply of machine learning engineers, hiring a team of experts to architect an ML system from scratch is simply not an option for most organizations. This disparity helps explain why a well-resourced tech company like Google benefits disproportionately from ML.

But over the past several years, a number of open source ML models — including the famous BERT model for understanding language, which Google released in 2018 — have started to change the game. The complexity of creating a model the caliber of BERT, whose aptly named “large” version has about 340 million parameters, means that few organizations can even consider quarterbacking such an initiative. However, because it’s open source, companies can now tweak that publicly available playbook to tackle their specific use cases.

To understand what these use cases might look like, consider a company like Medallia, a Moveworks customer. On its own, Medallia doesn’t possess enough data to build and train an effective ML system for an internal use case, like IT support. Yet its small data does contain a treasure trove of insights waiting for ML to unlock them. And by leveraging new techniques to glean these insights, Medallia has become more efficient, from recognizing which internal workflows need attention to understanding the company-specific language its employees use when asking for tech support.

Massive progress with small data

So here’s the trillion-dollar question: How do you take an open source ML model designed to solve a particular problem and apply that model to a disparate problem in the enterprise? The answer starts with transfer learning, which, unsurprisingly, entails transferring knowledge gained from one domain to a different domain that has less data.

For example, by taking an open source ML model like BERT — designed to understand generic language — and refining it at the margins, it is now possible for ML to understand the unique language employees use to describe IT issues. And language is just the beginning, since we’ve only begun to realize the enormous potential of small data.

Above: Transfer learning leverages knowledge from a related domain — typically one with a greater supply of training data — to augment the small data of a given ML use case.

Image Credit: Moveworks

More generally, this practice of feeding an ML model a very small and very specific selection of training data is called “few-shot learning,” a term that’s quickly become one of the new big buzzwords in the ML community. Some of the most powerful ML models ever created — such as the landmark GPT-3 model and its 175 billion parameters, which is orders of magnitude more than BERT — have demonstrated an unprecedented knack for learning novel tasks with just a handful of examples as training.

Taking essentially the entire internet as its “tangential domain,” GPT-3 quickly becomes proficient at these novel tasks by building on a powerful foundation of knowledge, in the same way Albert Einstein wouldn’t need much practice to become a master at checkers. And although GPT-3 is not open source, applying similar few-shot learning techniques will enable new ML use cases in the enterprise — ones for which training data is almost nonexistent.

The power of the collective

With transfer learning and few-shot learning on top of powerful open source models, ordinary businesses can finally buy tickets to the arena of machine learning. But while training ML with transfer learning takes several orders of magnitude less data, achieving robust performance requires going a step further.

That step is collective learning, which comes into play when many individual companies want to automate the same use case. Whereas each company is limited to small data, third-party AI solutions can use collective learning to consolidate those small data sets, creating a large enough corpus for sophisticated ML. In the case of language understanding, this means abstracting sentences that are specific to one company to uncover underlying structures:

Above: Collective learning involves abstracting data — in this case, sentences — with ML to uncover universal patterns and structures.

Image Credit: Moveworks

The combination of transfer learning and collective learning, among other techniques, is quickly redrawing the limits of enterprise ML. For example, pooling together multiple customers’ data can significantly improve the accuracy of models designed to understand the way their employees communicate. Well beyond understanding language, of course, we’re witnessing the emergence of a new kind of workplace — one powered by machine learning on small data.

Let’s block ads! (Why?)

Big Data – VentureBeat

To Improve Data Quality, Stop Playing the Data Telephone Game

October 3, 2020 TIBCO Spotfire

Reading Time: 3 minutes

Do you remember when you used to play the telephone game with other children? You know, the game where the first person in a chain whispers a phrase to the second, then the second repeats it to the third, and on until the last person repeats it back to the first.

It was such a joy to laugh about how what began as “The sun is in the sky” somehow transformed into “It is fun to eat pie” as the phrase passed from one friend to the next.

Further laughs quickly followed when each person in the chain shared their whispered phrase with all, allowing everyone to see what went wrong and where.

The Data Telephone Game

Interestingly, over the past thirty years, data management has adopted the same telephone game formula, copying data from one database to the next, with many stops on the journey.

Take the classic enterprise data warehouse process as an example:

The data starts as transaction records stored in a transaction system’s database
Next it moves from the source system to a staging database
From staging it moves to a data warehouse
With subsets of that data further advancing for storage within satellite data marts
Many of which soon feed individual Excel files resident on laptops

Or the more recent cloud data lake paradigm:

Source data from devices are consolidated on edge databases
This edge data is then copied into a cloud data lake for further analysis
Additional data from transaction systems might also be added to the lake
And to inject historical context, warehouse data might also be copied into the lake

Conceptually, these data management best-practices provide the opportunity to improve data quality by applying selected value-add transformations at various steps. But with so many rigid links in the chain, this data version of the telephone game can often inadvertently turn “sky” into “pie.” The business impact of this quality problem produces anything but childhood chuckles.

How Big is this Repeated Copying of Data Problem?

Just how much data is getting copied? In its Worldwide Global DataSphere Forecast 2019-2023, IDC estimates that for every terabyte of net-new data, over six additional terabytes of copied data is generated via replication and distribution. That is a lot of opportunities for “sky” to become “pie.”

Three Ways to Stop Playing the Telephone Game

IDC’s numbers, when combined with everyone’s telephone game experience, suggest trying a different approach to improve data quality. Here are three common-sense things organizations might consider.

Copy Less, Virtualize More – Data Virtualization is a proven method for integrating data without physically copying it. This will substantially reduce the transformation errors and entropy inherent in typical multiple-copy, data warehouse, and data lake deployments. Beyond fewer copies, data virtualization directly improves data quality via metadata-driven syntactic and semantic transformations and enrichments that standardize datasets and encourage reuse. Everyone is on the same page. And when things change, as they inevitably do, it’s a lot easier to modify centrally managed metadata definitions than it is to modify multiple distributed ETLs and database schemas.
Share Reference Data Everywhere – Reference Data Management improves data quality by enabling organizations to consistently manage standard classifications and hierarchies across systems and business lines. This lets them achieve needed consistency and compliance without extra copies. And by adding data virtualization as the distribution method, organizations can easily share and reuse reference data held in one virtual location.
Think Data Domain, Not Database Technology – Today, there are lots of cool, fit-for-purpose database technologies. But “new and exciting” doesn’t necessarily translate into “high business value.” Instead, think about the most valuable data domains. For example, if customer excellence is your competitive advantage, then focus on improving quality within the customer data domain. Master Data Management is the key to success in this case, allowing organizations to ensure data integrity within selected data domains such as customer, employee, product, and more.

Data Virtualization is a proven method for integrating data without physically copying it. This will substantially reduce the transformation errors and entropy inherent in typical multiple-copy, data warehouse, and data lake deployments. Click To Tweet

Stop Playing the Data Telephone Game

Let’s leave the telephone game to the kids. Instead, improve your data quality by executing the three common-sense recommendations above with TIBCO Unify. To learn more, talk to TIBCO and our partners.

Let’s block ads! (Why?)

The TIBCO Blog

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

September 30, 2020 Self-Service BI

This weekend I had the pleasure of presenting at SQL Saturday Copenhagen #963 with a session with the title “Power BI Paginated Reports – How to and why”.

I was asked if it was possible to create datadriven report subscriptions in paginated reports and I said “Not yet” – it turns out that is not exactly true.

With the new action “Export to File for Paginated Reports (preview)” – in Power Automate – link

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

It is possible !!!

And yes, for now we need the reports to be stored in a workspace backed by premium capacity but soon it will be available here

Setup data driven report subscriptions for #PowerBI paginated reports with #PowerAutomate

How to set it up