Business Intelligence Archives - Business Intelligence Info

Category Archives: Business Intelligence

Google updates Open Images to help improve app interface design

February 26, 2020 Big Data

Today, Google’s Open Images corpus for computer vision tasks got a boost with new visual relationships, human action annotations, and image-level labels, as well as a new form of multimodal annotations called localized narratives. Google says this last addition could create “potential avenues of research” for studying how people describe images, which could lead to interface design insights (and subsequently improvements) across web, desktop, and mobile apps.

Google introduced Open Images in 2016, a data set of millions of labeled images spanning thousands of object categories. Major updates arrived in 2018 and 2019, bringing with them 15.4 million bounding-boxes for 600 object categories and segmentation masks (which mark the outline of objects) for 2.8 million object instances in 350 categories.

“Along with the dataset itself, the associated Open Images [c]hallenges have spurred the latest advances in object detection, instance segmentation, and visual relationship detection,” wrote Jordi Pont-Tuset, a research scientist at Google Research. “Open Images is the largest annotated image dataset in many regards, for use in training the latest deep convolutional neural networks for computer vision tasks.”

As Pont-Tuset explains, one of the motivations behind localized narratives is to leverage the connection between vision and language, which is typically done via image captioning (i.e., images paired with written descriptions of their content). But image captioning lacks visual “grounding.” To mitigate this, some researchers have drawn bounding boxes for the nouns in captions after the fact — in contrast to localized narratives, where every word in the description is grounded.

The localized narratives in Open Images were generated by annotators who provided spoken descriptions of images while they hovered over regions they were describing with a computer mouse. The annotators manually transcribed their description, after which Google researchers aligned it with automatic speech transcriptions, ensuring that the speech, text, and mouse trace were correct and synchronized.

“Speaking and pointing simultaneously are very intuitive, which allowed us to give the annotators very vague instructions about the task.” explained Pont-Tuset. “[This latest version of] Open Images is a significant qualitative and quantitative step towards improving the unified annotations for image classification, object detection, visual relationship detection, and instance segmentation … [w]e hope that [it] will further stimulate progress towards genuine scene understanding.”

Open Images is freely available. According to Google, Open Images’ totals now stand at 9 million images annotated with 36 million image-level labels, 15.8 million bounding boxes, 2.8 million instance segmentations, 391,000 visual relationships, and 59.9 million human-verified image-level labels in over nearly 20,000 categories.

Let’s block ads! (Why?)

Big Data – VentureBeat

Google researchers use multiple cameras to reduce error rates for robot insertion and stacking tasks

February 25, 2020 Big Data

Object-manipulating robots rely on cameras to make sense of the world around them, but these cameras often require careful installation and ongoing calibration and maintenance. A new study published by researchers at Google’s Robotics division and Columbia University proposes a solution, which involves a technique that learns to accomplish tasks using multiple color cameras without an explicit 3D representation. They say that it achieves superior task performance on difficult stacking and insertion tasks compared with baselines.

This latest work builds on Google’s vast body of robotics research. Last October, scientists at the company published a paper detailing a machine learning system dubbed Form2Fit, which aims to teach a picker robot with a suction arm the concept of assembling objects into kits. Google Brain researchers are pursuing a novel robot task planning technique involving deep dynamics models, or DDM, that they claim enables mechanical hands to manipulate multiple objects. And more recently, a Google team took the wraps off of ClearGrasp, an AI model that helps robots better recognize transparent objects.

As the researchers point out, until recently, most automated solutions were designed for rigid settings where scripted robot actions are repeated to move through a predefined set of positions. This approach calls for a highly calibrated setup that can be expensive and time-consuming, and one that lacks the robustness needed to handle changes in the environment. Advancements in computer vision have led to better performance in grasping, but tasks like stacking, insertion, and precision kitting remain challenging. That’s because they require accurate 3D geometric knowledge of the task environment including object shape and pose, relative distances and orientation between locations, and other factors.

By contrast, the team’s method leverages a multi-camera view and a reinforcement learning framework that takes in images from different viewpoints and produces robot actions in a closed-loop fashion. By combining and learning directly from the camera views without an intermediary reconstruction step, they say it’s able to improve state estimation while at the same time increasing the robustness of the system’s actions.

In experiments, the researchers deployed their setup to a simulated environment containing a Kuka arm equipped with a gripper, two bins placed in front of the robot, and three cameras mounted to overlook those bins. The arm was first tasked with stacking one bin with a single block in a random position, starting with a single block either blue or orange in color. In other tasks, it had to insert a block firmly into a middle fixture and to stack blocks one on top of the other.

The researchers ran 180 data collection jobs across 10 graphics cards to train their reinforcement learning model, with each producing roughly 5,000 episodes per hour for the insertion tasks. They report it achieved success, with “large reductions” to error rates on precision-based tasks — specifically 49.18% on the first stacking task, 56.84% on the second stacking task, and 64.1% on the insertion task. “The effective use of multiple views enables a richer observation of the underlying state relevant to the task,” wrote the paper’s coauthors. “Our multi-view approach enables 3D tasks from RGB cameras without the need for explicit 3D representations and without camera-camera and camera-robot calibration. In the future, similar multi-view benefits can be achieved with a single mobile camera by learning a camera placement policy in addition to the task policy.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Microsoft Azure Sphere launches in general availability

February 24, 2020 Big Data

In April 2018, nearly two years ago, Microsoft announced Azure Sphere, a program to better secure the 41.6 billion internet for things (IoT) devices expected to be connected to the internet by 2025. Now, following a lengthy preview, the tech giant is this week launching Azure Sphere in general availability.

Eligible customers will be able to sign up in the coming days. Azure Sphere doesn’t have ongoing fees associated with it, but there’s a one-time cost for a chip (as little as less than $ 8.65) that includes access to all of Sphere’s components, plus OS updates for the lifetime of the chip. Alternatively, developers can license Visual Studio and Microsoft’s Azure IoT services to develop apps for Sphere “more efficiently, according to Azure IoT CVP Sam George.

“We live in an increasingly connected world,” added George, who noted last year that Microsoft’s Azure IoT software-as-a-service (SaaS) suite grew nearly 150% year-over-year from 2018 to 2019 and gained over 100 new features. “At Microsoft, we are committed to providing a trusted, easy-to-use platform that allows our customers and partners to build seamless, smart, and secure solutions regardless of where they are in their IoT journey.”

Azure Sphere

For the uninitiated, Azure Sphere is a high-level software-as-a-service (SaaS) platform with built-in communication features for cross-industry IoT devices. It comprises integrated hardware built around a secured silicon chip; the Azure Sphere OS, a custom Linux-based operating system; and the Azure Sphere Security Service, a cloud-based service that provides continuous security.

The Azure Sphere OS runs on Azure Sphere-certified chips and connects to the Azure Sphere Security Service, and it’s designed to provide a platform for IoT app development — including both high-level and real-time capable apps. It’s the first operating system running a Linux kernel and the second Unix-like operating system that Microsoft has publicly released, interestingly, the other being the decades-old and discontinued Xenix.

Microsoft Azure Sphere launches in general availability

Above: Avnet’s Azure Sphere MT3620 starter kit, which packs a MediaTek chip.

Image Credit: Avnet

Azure Sphere-certified chips and hardware support two implementations: greenfield and brownfield. Greenfield involves designing and building IoT devices with a supported (and optionally cellular-enabled) chip produced or soon-to-be-produced by MediaTek, NXP, and Qualcomm. As for brownfield, it calls for an Azure Sphere “guardian” device — the bulk of which are produced by Avnet and AI-Link — to connect existing hardware to the internet.

Azure Sphere hardware is available in several configurations, including Wi-Fi modules, development kits and dev boards (from partners like Seeed Studios and USI), and the aforementioned guardian. Speaking of, the guardian module — which doesn’t directly connect to networks — is a peripheral with a built-in Azure Sphere-certified chip that runs the Azure Sphere OS and the Azure Sphere Security Service, both of which can be configured via a Wi-Fi or Ethernet connection.

Security

A core component of Azure Sphere is the above-mentioned Security Service, a cloud-based service that enables maintenance, updates, and control for Azure Sphere-certified chips. It establishes a connection between devices and the internet and various ancillary cloud services and ensures secure boot, authenticating device identity, integrity, and root of trust while certifying the device is running a vetted codebase. The Azure Sphere Security Service additionally provides the channel by which Microsoft automatically downloads and installs Azure Sphere OS updates and app updates across deployed devices.

At the hardware level, complementing the Security Service, there’s Pluton. It’s a Microsoft-designed security subsystem that implements root of trust for Azure Sphere via a combination of techniques:

A custom-designed security processor core
Cryptographic engines
A hardware-based random number generator
A public and private key generator
Asymmetric and symmetric encryption
An elliptic curve digital signature algorithm verification for secure boot
Measured boot in silicon to support remote attestation with a cloud service
Various tampering counter-measures

App development

App development on Azure Sphere OS is relatively straightforward. Using the Azure Sphere SDK for Linux or Windows and samples and solutions open-sourced on GitHub, developers can deploy apps that make use of peripherals on Azure Sphere-certified chips. The apps in question run atop a primary processor core with access to external networking or a lower-powered core as a real-time capable app, with real-time capable apps running either on bare metal or with a real-time operating system, and they can be distributed to Azure Sphere devices through the same mechanism as Azure Sphere OS updates.

Of course, development isn’t confined to Azure. The chips work with other public, private, and hybrid cloud environments including Amazon Web Services and Google Cloud, which no doubt appealed to Azure Sphere’s early adopters.

Here’s a few of them:

Microsoft’s own datacenter team tapped Azure Sphere guardian modules to connect equipment and systems for the first time and build new systems.
Qiio developed an Azure Sphere-based IoT deployment solution that combines hardware, cellular connectivity, and cloud services.
Vitamix incorporated Azure Sphere into its IoT Module, a retrofit device that allows users to remotely program Vitamix blenders.
Elettrone is in the process of building an Azure Sphere energy monitoring solution to reduce waste in commercial and residential properties.
Starbucks partnered with Microsoft to deploy Azure Sphere across its existing equipment in stores globally using guardian modules.
Gojo, the brand behind Purell, plans to integrate Azure Sphere with motion detectors and connected dispensers in healthcare facilities.
Leoni, which develops cable systems for the automotive sector and other industries, uses Azure Sphere with integrated sensors to actively monitor cable conditions, creating intelligent and connected cable systems.

Microsoft in IoT

In 2018, Microsoft committed $ 5 billion to intelligent edge innovation by 2022 (an uptick from the $ 1.5 billion it spent prior to 2018) and pledged to grow its IoT partner ecosystem to over 10,000. It’s borne fruit in Azure IoT Central, a cloud service that enables customers to quickly provision and deploy IoT apps, and IoT Plug and Play, which provides devices that work with a range of off-the-shelf solutions. Microsoft’s investment has also bolstered Azure Sphere; Azure Security Center, its unified cloud and edge security suite; and Azure IoT Edge, which distributes cloud intelligence to run in isolation on IoT devices directly.

Microsoft has competition in Google’s Cloud IoT, a set of tools that connect, process, store, and analyze edge device data. Not to be outdone, Amazon Web Services’ IoT Device Management tracks, monitors, and manages fleets of devices running a range of operating systems and software. And Baidu’s OpenEdge offers a range of IoT edge computing boards and a cloud-based management suite to manage edge nodes, edge apps, and resources such as certification, password, and program code.

But the Seattle company has ramped up its buildout efforts as of late, most recently with the acquisition of Express Logic, a San Diego, California-based developer of real-time operating systems (RTOS) for IoT and edge devices powered by microcontroller units. Separately, it’s partnered with companies like DJI, SAP, PTC, Qualcomm, and Carnegie Mellon University for IoT and edge app development.

Let’s block ads! (Why?)

Big Data – VentureBeat

SQL Server Machine Learning 2019: Working with Security Changes

February 24, 2020 BI News and Info

I have a confession to make. Why, in my last article about shortest_path in SQL Server 2019, have I used Gephi in order to illustrate the relationships, instead of using a script in R for the same purpose and demonstrate Machine Learning Services as well?

The initial plan was to use an R script; however, the R script which works perfectly in SQL Server 2017 doesn’t work in SQL Server 2019.

This is the original R script I had planned to use:

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘ require(igraph)

g <- graph.data.frame(graphdf)

V(g)$ label.cex <- 2

png(filename = “c:\R\plot1.png”, height = 1200, width = 1200, res = 100);

plot(g, vertex.label.family = “sans”, vertex.size = 40)

dev.off() ‘,

@input_data_1 = N‘select LikeMember.MemberName as LikeMember,

LikedMember.MemberName as LikedMember

from dbo.ForumMembers as LikeMember, dbo.ForumMembers as LikedMember,

Likes

where Match(LikeMember-(Likes)->LikedMember)’,

@input_data_1_name = N‘graphdf’

The purpose of the script was only to create an image file with the relationships stored in graph tables, but in SQL Server 2019, it results in an access denied message.

If you would like to reproduce this demonstration, you can use this script file to create the database in a SQL Server 2017. You must also install the R packages needed by the script. There are many ways to do it. My favourite is this:

Open windows explorer
Browse to the SQL Server R folder, usually is this one: C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\R_SERVICES\bin
Execute the R.EXE application
Execute the instruction install.packages(“igraph”) , where “igraph” is the name of the package to be installed

When you try and run the above R script. It will work well in SQL Server 2017, and fail in SQL Server 2019 with the error message you can see in the image below:

a screenshot of a cell phone description automati 14 SQL Server Machine Learning 2019: Working with Security Changes

During PASS Summit, I was able to ask questions to the Microsoft employees in the Data Clinic, and they explained the differences between the two versions. The explanation hasn’t solved all the problems, but I was able to find a solution and create a workaround for this script.

While researching the workaround, I saw many examples of R scripts manipulating files and all these examples may fail in SQL Server 2019. Highlighting this problem, solution, and workaround in this article will be important to everyone using SQL Server Machine Learning and planning to migrate to SQL Server 2019.

First, I’ll review how Machine Learning in SQL Server works: it involves the execution of an external script, in R or Python, but has a flexible structure, you could create your own external language. The execution itself is started by a service called SQL Server Launchpad, isolating the execution from SQL Server itself. The image below (from What is the Machine Learning Server) illustrates how the execution architecture works, although this article will not go so deep into the architecture.

a screenshot of a cell phone description automati 15 SQL Server Machine Learning 2019: Working with Security Changes

Launchpad in SQL Server 2017

In SQL Server 2017 the Launchpad starts processes for each execution. The processes need an identity, so SQL Server dynamically creates users inside the group SQLRUserGroup. In the following image, you can see two groups, SQLRUserGroup, for SQL Server 2019 and SQLRUserGroupSQL2017 for SQL Server 2017, since I have both in the same machine.

The difference in the name is due to the instance name; SQL Server 2019 is the default instance in my machine. Due to that, its user group name has only the core name, while SQL Server 2017 is in an instance called SQL2017, so the instance name is attached to the core group name.

The processes have access to the machine resources, limited by the permissions given to the SQLRUserGroup. Since SQL Server 2017 creates many users, you shouldn’t grant permissions directly to the users, only to the group.

In order to be able to manipulate files in the file system, you only need to give the correct permissions to the group SQLRUserGroup. The image below shows the group and users for SQL Server 2017

a screenshot of a cell phone description automati 16 SQL Server Machine Learning 2019: Working with Security Changes

Launchpad in SQL Server 2019

In SQL Server 2019, the Launchpad was improved. Instead of using processes for each execution, it uses app containers. The app containers need only a single identity, so they use the same identity as the Launchpad service.

While the 2017 version, the SQLRUserGroup, had many users; in the 2019 version, it has only one, the Launchpad service account. The image below shows the group for SQL Server 2019.

a screenshot of a cell phone description automati 17 SQL Server Machine Learning 2019: Working with Security Changes

Nowadays, when talking about containers immediately, we think about Docker, but not in this case. The app containers I’m mentioning here are a sandbox set of APIs introduced in Windows 8. These allow any application to create app containers and start an execution inside an app container.

The app containers work as isolated virtual machines (but much more lightweight, of course). This means that you can’t access the file system of the host (your SQL Server), neither for reading or to save files. You save any files inside the app container, and they are destroyed after the app container is released. In fact, the container is redirected to save files in a temporary folder inside the host hard drive. This increases the security, no doubt about that, but any script dealing with the file system may need changes to work in SQL Server 2019.

These app containers are created by Launchpad using the windows Sandbox API. The Launchpad doesn’t offer any way to customize the app containers creation, which could solve the problem.

You can learn more about app containers with these two links:

https://techcommunity.microsoft.com/t5/Windows-Kernel-Internals/Windows-Sandbox/ba-p/301849
https://www.malwaretech.com/2015/09/advanced-desktop-application-sandboxing.html

Proving this Concept

In order to determine this concept, you can use a script in R to access the disk and list the files. You can execute the script below in both SQL Server versions, 2017 and 2019, and compare the results. In SQL Server 2017, the script will have access to the host disk. In SQL Server 2019, the script will have access only to a fake disk, which is, in fact, a temporary folder in the host disk and will be deleted after its use by the app container.

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘data <- list.files()

data2 <- data.frame(data)’,

@output_data_1_name = N‘data2′

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘data <- list.dirs(“..\”)

data2 <- data.frame(data)’,

@output_data_1_name = N‘data2′

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘data <- list.dirs(“..\..”)

data2 <- data.frame(data)’,

@output_data_1_name = N‘data2′

The image below the script shows the result in SQL Server 2019.

Solutions

The app containers are created during the SQL Server Machine Learning services setup. They are objects inside the Windows local directory, and such as all the objects in the local directory, they have a unique SID to identify the app container.

You can define file system permissions directly for the SIDs. The problem is: How to identify the SID’s of the app containers since they are not listed as user or groups.

Analyse two possible solutions:

Easy and tempting: You can give permission to the object called All Application Containers. The R scripts will have the file system permission; however, any other app container eventually running on the same machine will have the file system permission as well.
Secure: You can identify the SIDs of the app containers installed by SQL Server and give permissions directly to them. In this way, only the app containers used by the SQL Server Machine Learning Services will receive these permissions

You can test both solutions using a straightforward script. First, create a folder called C:\testFolder and copy some files, any files, to the folder. Try to list the files in this folder using the script below.

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘data <- list.files(“c:\testFolder”)

data2 <- data.frame(data)’,

@output_data_1_name = N‘data2′

All Application Containers

The All Application Container object has a fixed SID in the local directory, which is S-1-15-2-1 . You can use the application icacls to grant permission

Execute the R script above against the 2019 instance using SSMS. The script will return no result
Press Win+R and type CMD to open a command prompt as administrator
Type the following instruction and press ENTER

icacls c:\testFolder /grant *S-1-15-2-1:(OI)(CI)F /t

Using Windows Explorer, right-click the testFolder and select Properties in the context menu
In the Properties window, click the Security tab. You will be able to confirm the permission, like the image below.

Open SQL Server Configuration Manager
On the left side of SQL Server Configuration Manager, select SQL Server Services
In the right-side of SQL Server Configuration Manager, right-click the Launchpad service and select Restart item in the context menu
Execute the same R script again in SSMS. This time it will work and list the files in the folder.

a screenshot of a cell phone description automati 18 SQL Server Machine Learning 2019: Working with Security Changes

Delete the testFolder folder so that you can continue with the other solution below

Using the App Containers SID

The challenge you will face when giving permissions to the App Containers SID is to discover which are the App Containers SID. There is not a direct solution for that, although, after knowing where to look, it becomes easy. SQL Server installs firewall rules in Windows Firewall in order to forbid app containers from making external contact to the network. These firewall rules are created precisely to block the app containers SID from using the network, so you can identify these SIDs by analysing these firewall rules.

If you look at the firewall rules, you can see that the app containers are blocked.

word image 81 SQL Server Machine Learning 2019: Working with Security Changes

The firewall UI doesn’t provide the SIDs, however. You need to dig deeper using PowerShell to retrieve the SIDs from the firewall rules.

The PowerShell command is a combination of the cmdlet Get-NetFirewallRule, to retrieve the firewall rules and the cmdlet Get-NetFirewallApplicationFilter in order to retrieve, from each rule, the app container filter information.

The complete PowerShell command line is this:

(Get-NetFirewallRule | Where-Object { ($ _.Direction -eq “Outbound”) -and ($ _.DisplayName -like “*appcontainer*”)}) | %{ ($ _ | Get-NetFirewallApplicationFilter) | %{Write-Output $ _.Package } }

Analyse this command line in more detail:

Get-NetFirewallRule will retrieve all the firewall rules in Windows Firewall
Where-Object filters the firewall rules, retrieving only the outbound rules which contains ‘appcontainer’ in the name
The expression %{ } is a shortcut to foreach-object so that the instructions will be executed for each firewall rule
Get-NetFirewallApplicationFilter is executed for each firewall rule, having the firewall rule ($ _ ) as a parameter
The expression % { } is used again to run one line for each application filter found
The Write-Output shows the value of the Package property on each application filter

a black and blue text description automatically g 1 SQL Server Machine Learning 2019: Working with Security Changes

Having discovered the SIDs, you need to set the permission to the folder. The instruction will look like the one below for each SID.

ICACLS “C:\testFolder” /grant “*S-1-15-2-1853514363-3573294594-1452771951-225645021-3819702349-824866239-3302992986″:(OI)(CI)F /t

You could copy and paste all the sids returned to create a batch file to set the permissions, but PowerShell can do all of this you as well. The following script prompts you for a file path, the one where you want Machine Learning to have access, and sets the permissions:

Param (

[Parameter(

Mandatory)]

[string]

$ FolderName

)

Get-NetFirewallRule |

Where-Object {

$ _.Direction -eq “Outbound” -and

$ _.DisplayName -like “*appcontainer*”

} |

ForEach-Object {

$ _ |

Get-NetFirewallApplicationFilter |

ForEach-Object {

$ Acl = Get-Acl $ FolderName

$ Sec= New-Object System.Security.Principal.SecurityIdentifier($ _.Package)

$ AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule(

$ Sec,

“FullControl”,

(1 -bor 2),

“Allow”

)

$ Acl.SetAccessRule($ AccessRule)

$ Acl |

Set-Acl $ FolderName

} # end 2nd FE-O

} # end 1st FE-O

The script is using COM objects System.Security.Principal.SecurityIdentifier and System.Security.AccessControl.FileSystemAccessRule in order to set the access rule on the folder.

After running the script, take a look at the security properties of the folder, and the R script should now work from SQL Server 2019.

word image 82 SQL Server Machine Learning 2019: Working with Security Changes

Workaround with a FileTable

Although I have identified two solutions, it’s interesting to notice Microsoft is moving SQL Server Machine Learning towards a more secure environment, avoiding network and file system access. Considering this, you may like to use an alternative solution.

There are some options, all based on returning the image as an output parameter to SQL Server. Here is one of the options.

A filetable can map a disk folder as a table in SQL Server. This a very good solution when you need to deal with files. The script in R will see the table as a regular table. While reading or inserting into the table, the script will be reading and inserting from/to the disk.

It’s a very good solution for a permanent environment, however, it has some problems:

You can’t define the final location of each file using SQL Server configuration. The filetable will be exposed from the server as a sharing folder, not a disk folder, so there is a slight difference in relation to the initial planning.
Filetable is not supported in Azure PaaS cloud solutions, so you will be tied to IaaS solutions, which means creating and maintaining a virtual machine in Azure, or an on-premise environment.

The Filetable technology was created in SQL Server 2016, based on the filestream technology which was built on the previous version.

Enabling Filetable in SQL Server

A filetable solution needs some administrative care; for this reason, this solution can only be used after enabling it on a server configuration level. Follow these steps to enable this solution:

Open SQL Server Configuration Manager
On the left side of SQL Server Configuration Manager, select SQL Server Services
On the right side, double click the SQL Server service you are using
On the SQL Server properties window, select the FileStream tab

a screenshot of a cell phone description automati 19 SQL Server Machine Learning 2019: Working with Security Changes

In the filestream tab, check the box Enable filestream for Transact-SQL Access
In the filestream tab, check the box Enable filestream for I/O Access
The textbox Windows Share Name will be filled with the default value MSSQLSERVER. You can change it, but leave this default for now
In the filestream tab, check the box Allow remote clients to access filestream data.
Click the Ok button in the SQL Server Properties window
In the right-side of SQL Server Configuration Manager, right-click the SQL Server service and select Restart item in the context menu

a screenshot of a cell phone description automati 20 SQL Server Machine Learning 2019: Working with Security Changes

In SSMS, change the FileStream access level configuration using the following code:

EXEC sp_configure ‘filestream access level’, 2

RECONFIGURE

Value 1 enables filestream only for T-SQL access, which is not enough for filetable. Value 2 enables filestream for T-SQL and I/O access.

Preparing the Database to Support Filetable

In order to support filetable, you need to make some changes to the database. You need to create a filegroup to contain the filestream data, configure a special file, which will point to a folder and change some database settings.

Here is the T-SQL to execute for the database. Be sure to delete the c:\R folder before running this code:

ALTER DATABASE graphdemo ADD filegroup filestreamgroup

CONTAINS filestream

ALTER DATABASE graphdemo ADD FILE (NAME=fs, filename=‘c:\R’)

TO filegroup filestreamgroup

In SSMS, Object Explorer, right-click the GraphDemo database and select Properties in the context menu

In the database properties window, set the property Filestream Directory Name to RFiles. During further steps, you will better understand what this means.
In the database properties window, set the property Filestream Non-Transaction access to Full

In the database properties window, click the Ok button
Confirm the close of database connections clicking the Ok button in the dialog box that will appear

a screenshot of a cell phone description automati 21 SQL Server Machine Learning 2019: Working with Security Changes

Create the filetable using the following T-SQL:

CREATE TABLE images AS filetable WITH

(

filetable_directory = ‘Images’

);

The filetable you created, Images, is linked to a folder. Which folder? If you check the path c:\R, which was used to configure the database file, you will find some files and folders, but it’s not easy to understand them. So, what’s missing?

Follow these steps to see:

Click win+R
Type \localhost and click Ok You will find the share MSSQLSERVER, that’s the name you configured in SQL Server Configuration Manager. Now that you know how the name is used, you can choose a better name if needed.

Open the share MSSQLSERVER You will find a folder called RFiles. That’s the configuration made for the database. Each database configured to use filestream will have a share inside \Localhost\MSSQLSERVER

a screenshot of a cell phone description automati 22 SQL Server Machine Learning 2019: Working with Security Changes

Open the folder Rfiles You will find another folder called Images, the name used for the filetable. Each filetable in the database GraphDemo will have its own folder inside \localhost\MSSQLSERVER\RFILES

a screenshot of a cell phone description automati 23 SQL Server Machine Learning 2019: Working with Security Changes

Open the folder Images. The folder is empty.
In SSMS, execute the following query:

select * from Images

The filetable is empty.
Copy a file, any file, to the Images folder, \localhost\MSSQLServer\RFILES\Images
Execute the select again in SSMS:

select * from Images

This time the table has one row, exactly the row for the new file you copied. The filetable is linked with the folder. So, if you insert a record inside the filetable, the record will be a file available in the folder, in this way you can save to disk the image generated by the R script

New R Script for Filetable

After preparing the filetable environment, the new script will be this one:

DECLARE @img VARBINARY(max)

EXEC Sp_execute_external_script

@language = N‘R’,

@script = N‘ require(igraph)

require(hexView)

g <- graph.data.frame(graphdf)

V(g)$ label.cex <- 2 png(filename = “plot1.png”, height = 1200, width = 1200, res = 100);

plot(g, vertex.label.family = “sans”, vertex.size = 40)

dev.off()

img <- readRaw(“plot1.png”)

imageContent <- img$ fileRaw’,

@input_data_1 = N‘select LikeMember.MemberName as LikeMember,

LikedMember.MemberName as LikedMember

from dbo.ForumMembers as LikeMember, dbo.ForumMembers as LikedMember,

Likes

where Match(LikeMember-(Likes)->LikedMember)’,

@input_data_1_name = N‘graphdf’,

@params = N‘@imageContent varbinary(max) OUTPUT’

@imageContent = @img output;

INSERT INTO images (NAME, file_stream)

VALUES (‘test.png’, @img)

The script is very similar to the initial one; however, this time, it results in an output variable with the image.

Here are the differences between this script and the original:

A varbinary(max) variable which will be used as an output
A new R library, hexView, must be installed in order to do additional manipulations in the file (yes, you will still use a file)
After saving the plot to disk, it’s read back, reading the raw file content
The variable where you stored the raw image (imageContent) is defined as a parameter using the @params argument of the sp_execute_external_script
You need to link this script variable with a variable outside the script to get the content
Finally, you insert the variable into the filetable

The result of the execution of the script above will be a new file in the folder, you can check by accessing \localhost\MSSQLSERVER\RFILES\Images

a screenshot of a cell phone description automati 25 SQL Server Machine Learning 2019: Working with Security Changes

Conclusion

During my research on the web, I saw many examples of R scripts dealing with the file system. These scripts which are running nowadays in SQL Server 2017 will break in SQL Server 2019 unless one of the changes explained in this article is applied.

You can choose to fix the file system permission or start to avoid direct file system access from the R script. I believe Microsoft will provide a fix for this problem soon, what probably will be an easier way to give permissions to the app container SIDs

Let’s block ads! (Why?)

SQL – Simple Talk

Want to get more from your data? Stop focusing on efficiency

February 23, 2020 Big Data

Ever since Henry Ford came up with the idea of using moving assembly lines to build automobiles faster at lower cost and at higher quality, efficiency has been the driving force in industry.

Unfortunately, it’s the wrong approach for modern businesses. We’re no longer in an industrial economy. Today, information powers our world. Efficiency drives profit in the short term, but making it your primary focus doesn’t drive innovation; it opens the door for competitors to provide something better.

Despite this, we’ve applied the efficiency mantra to almost every aspect of business. The primary use of data in many organizations is to cut costs and increase profitabilty for existing products. Dashboards and reports are really tools to answer questions like: Where is our process breaking? Which parts of our business are the least efficient? Where can we reduce cost? Data is the language of that conversation.

But the focus on efficiency is limiting. Businesses have more opportunities in environments of plenty, fewer in environments of scarcity. Most businesses today exist in a world of plenty: We have new ways to reach customers, like smartphones, chatbots and global logistics networks, and more ways to innovate, like machine learning, 5G, and an abundance of data and cheap processing power.

Using fewer resources, which is the focus of an efficiency program, makes us lean and mean, but it doesn’t provide fertile ground for growth. It can even be counterproductive. Call times can only be so low while still maintaining acceptable customer service. After a certain point, productivity gains are offset by quality issues.

Does this mean we should throw efficiency off a cliff? Obviously not. But using data smartly can provide the best of both worlds. Efficiency is appropriate for non-differentiating, mature cost centers, but it shouldn’t be the prevailing corporate philosophy. To succeed in today’s fast-moving tech landscape, the focus has to be innovation.

That’s all well and good, but how do we apply data to drive innovation instead of efficiency? I have three recommendations.

Think outside your business

Data isn’t just for internal stakeholders. Increasingly, companies realize that providing data externally is a vital part of their business. Whether it’s providing billing detail to business customers, transaction detail to consumers, or performance data to vendors, data is increasingly a differentiator. Making it available externally is a great first step away from an efficiency focus.

Get data to business users, not just technical users

Historically, access to data has been limited to technical people who speak the language of data, but these are rarely the same people who own the business outcome. Business users need wider access to data, which will allow them to think of new uses that only they can identify because they understand the customers and the business.

Opening up access to data is becoming easier. Modern analytics tools allow non-technical users to experiment with data in creative ways. To enable this, make sure the people who can innovate with data have access to it — not just the analysts.

Optimize for maturity, innovate for growth

It’s important to distinguish among different parts of the business. Efficiency should be applied in the mature areas. Use data to identify segments that have slow or declining growth and apply efficiency there. Everything else should be managed through the lens of innovation.

That applies especially to areas of differentiation — and these can vary even within the same industry. For example, a bank may view its customer call center as simply a cost center, so outsourcing those services may be appropriate. Another bank may see customer service as its competitive advantage. It should use data to identify its best customers, predict their future value, and route them to the best agents.

We’re in a remarkably different economy from the world of Henry Ford. His laser focus on speed and cost were appropriate at the time, and still are for some traditional industries. But for most of us, innovating quickly is a prerequisite to survival. That means loosening the chokehold that efficiency has on our thinking and making data-driven innovation the driving force in business.

Doug Bordonaro is Field CTO at ThoughtSpot.

Let’s block ads! (Why?)

Big Data – VentureBeat

Write the code to display a given plot [closed]

February 23, 2020 BI News and Info

Write the code to display the exact image shown above.

2 Answers

Let’s block ads! (Why?)

Recent Questions – Mathematica Stack Exchange

Self-driving car companies complain California test data may mislead

February 23, 2020 Big Data

(Reuters) — Developers of self-driving cars are amping up criticism of a California reporting requirement on test data, saying the data could mislead, as the state prepares to release the latest results for 2019.

Companies such as General Motors’ Cruise and startup Aurora have said the metric, called disengagements, is not an accurate or relevant way to measure their technical progress, even though it is widely used to do just that. The debate is taking on more importance amid delays in the rollout of self-driving vehicles and concerns over a lack of regulation and the prospects for profitability for the companies that make such vehicles. The focus on disengagements — when a human driver must take manual control from a self-driving system — and the backlash from self-driving companies have been growing since the California Department of Motor Vehicles began releasing annual disengagement reports five years ago.

California requires all companies testing self-driving vehicles on public roads to submit an annual report on disengagements and what caused them, “written in plain language.” In 2018, the companies with the most miles between disengagements were Alphabet’s Waymo and Cruise. Companies with the greatest number of disengagements were Apple and Uber. In a February 2019 blog post, Waymo wrote that “the key to self-driving technology safely improving and scaling is through a robust breadth of experience and scenario testing, represented by a wider array of data points beyond disengagement alone.” In the same post, Waymo noted its disengagement rate in 2018 dropped 50% from the previous year, while miles between disengagements nearly doubled.

Self-driving companies say the disengagement data can draw unfair comparisons between companies and their self-driving technology. Aurora co-founder Chris Urmson, who previously headed Waymo’s self-driving program, wrote last month that “these numbers mean little when there’s no clear definition of what constitutes a disengagement.”

Cruise co-founder Kyle Vogt expressed similar concerns in a blog post last month, adding: “The general public and regulators deserve hard, empirical evidence that an (autonomous vehicle) has performance that is super-human” — that is, better than the average human driver — if the deployment of that technology is to have “a positive overall impact on automotive safety and public health.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Google launches TensorFlow library for optimizing fairness constraints

February 22, 2020 Big Data

Google AI today released TensorFlow Constrained Optimization (TFCO), a supervised machine learning library built for training machine learning models on multiple metrics and “optimizing inequality-constrained problems.”

The library is designed to help address issues like fairness constraints and predictive parity and help machine learning practitioners better understand things like true positive rates on residents of certain countries, or recall illness diagnoses depending on age and gender.

In tests with a Wikipedia data set, the library achieved lower false-positive rates when predicting whether a comment on a Wiki is toxic based on race, religion, gender identity, or sexuality, while maintaining similar accuracy rates.

TFCO is made to “take into account the societal and cultural factors necessary to satisfy real-world requirements,” said Andrew Zaldivar on behalf of the TFCO team today in a Google AI blog post.

“The ability to express many fairness goals as rate constraints can help drive progress in the responsible development of machine learning, but it also requires developers to carefully consider the problem they are trying to address,” he said. “A ‘safer’ alternative is to constrain each group to independently satisfy some absolute metric, for example by requiring each group to achieve at least 75% accuracy. Using such absolute constraints rather than relative constraints will generally keep the groups from dragging each other down.”

The library, which includes an optional “two-dataset” approach to improving generalization, is built on a trio of research papers published last year, according to the library GitHub page.

The release of the TFCO library comes a day after Google removed gendered labels from the Cloud Vision API.

Let’s block ads! (Why?)

Big Data – VentureBeat

Google Cloud AI removes gender labels from Cloud Vision API to avoid bias

February 21, 2020 Big Data

Google Cloud AI is removing the ability to label people in images as “man” or “woman” with its Cloud Vision API, the company told VentureBeat today. Labeling is used to classify images and train machine learning models, but Google is removing gendered labels because it violates Google’s AI principle to avoid creating biased systems.

“Given that a person’s gender cannot be inferred by appearance, we have decided to remove these labels in order to align with the artificial intelligence principles at Google, specifically Principle #2: avoid creating or reinforcing unfair bias. After today, a non-gendered label such as ‘person’ will be returned by Cloud Vision API,” a Google spokesperson told VentureBeat in an email.

VentureBeat reached out to Microsoft’s Azure and Amazon’s AWS to ask if they intend to remove gender labels from their cloud AI services. We will update this story with more information when we hear back.

The Google Cloud Vision API provides computer vision for customers to detect objects and faces. Google previously blocked the use of gender-based pronouns in an AI tool in 2018.

Many facial analysis and facial recognition systems on the market today predict gender but have challenges identifying people who do not conform to gender norms, people who are transgender, and women of color.

A study last fall by University of Colorado, Boulder researchers found that AI from Amazon, Clarifai, Microsoft, and others maintained accuracy rates above 95% for cisgender men and women but misidentified trans men as women 38% of the time. People with no gender identity were misidentified 100% of the time.

Lead author Morgan Klaus Scheuerman told VentureBeat he believes Google is attempting to set itself apart from competitors. The analysis found that Google did not provide gender classification services but did allow “man” and “woman” labels. Systems from companies like Microsoft can label people like waitresses, air woman, or military woman.

“We basically discussed [in work last fall] how the decisions that are being made in all systems are inherently political. And in the cases of where you’re kind of classifying things about human beings, it becomes more. I think we should assess more what the political notions of that are. And so I’m very excited that Google is kind of taking that seriously,” he told VentureBeat in a phone interview.

In recent years, researchers like Joy Boulamwini performing system audits found major facial recognition providers tend to work best on white men and worse on women of color.

A lack of high performance for all people is a primary reason lawmakers in state legislatures, cities like San Francisco, and the U.S. Senate have proposed bans or moratoriums on the use of facial recognition systems.

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers propose data set to measure few-shot learning performance

February 20, 2020 Big Data

A plethora of AI models have been tailored to tackle few-shot classification, which refers to learning a classifier for new classes given only a few examples (an ability humans naturally possess). Improving on it could lead to more efficient algorithms capable of expanding their knowledge without requiring large labeled data sets, but to date, many of the procedures and corpora used to assess progress here are lacking.

That’s why researchers at Google AI, the University of California, Berkeley, and the University of Toronto propose in a preprint paper a benchmark for training and evaluating large-scale, diverse, and more “realistic” few-shot classification models. They say it improves upon previous approaches by incorporating multiple data sets of “diverse” distributions and by introducing realistic class imbalance, which they say allows the testing of robustness across a spectrum from low-shot learning onward.

The work was published in May 2019, but it was recently accepted to the International Conference on Learning Representations (ICLR) that’ll be held in Addis Ababa, Ethiopia in April.

As the team explains, as opposed to synthetic environments, real-life learning experiences are heterogeneous in that they vary by the number of classes and the examples per class. They also measure only within-corpus generalization, and they ignore the relationships between classes when forming episodes — i.e., the coarse-grained classification of dogs and chairs may present different difficulties than the fine-grained classification of dog breeds. (An “episode” encompasses states that come between an initial-state and a terminal-state, such as a game of Chess.)

By contrast, the researchers’ data set — the Meta-Dataset — leverages data from 10 different corpora, which span a variety of visual concepts natural and human-made and vary in the specificity of the class definition. Two are reserved for evaluation, meaning that no two classes from them participate in the training set, while the remaining ones contribute some classes to each of the training, validation, and test splits of classes.

Meta-Dataset separately employs an algorithm for sampling episodes, which aims to yield imbalanced episodes of variable shots (class precision) and ways (accuracy). A prescribed number of examples of each chosen class are chosen uniformly at random to populate the support and query sets.

In experiments, the team trained meta-learning models via training episodes sampled using the same algorithm as used for Meta-Dataset’s evaluation episodes. They say that, tested against Meta-Dataset, the models generally didn’t improve when provided multiple data sources and that they didn’t benefit from meta-learning across the data sets. Moreover, they report that the models weren’t robust to the amount of data in test episodes; rather, each excelled in a different part of the spectrum.

“We believe that our exploration of various models on Meta-Dataset has uncovered interesting directions for future work pertaining to meta-learning across heterogeneous data,” wrote the coauthors, who added that addressing the uncovered shortcomings constitutes an important research goal. “[I]t remains unclear what is the best strategy for creating training episodes, the most appropriate validation creation and the most appropriate initialization.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Category Archives: Business Intelligence

Google updates Open Images to help improve app interface design

Google researchers use multiple cameras to reduce error rates for robot insertion and stacking tasks

Microsoft Azure Sphere launches in general availability

Azure Sphere

Security

App development

Microsoft in IoT

SQL Server Machine Learning 2019: Working with Security Changes

Launchpad in SQL Server 2017

Launchpad in SQL Server 2019

Proving this Concept

Solutions

All Application Containers

Using the App Containers SID

Workaround with a FileTable

Enabling Filetable in SQL Server

Preparing the Database to Support Filetable

New R Script for Filetable

Conclusion

Want to get more from your data? Stop focusing on efficiency

Think outside your business

Get data to business users, not just technical users

Optimize for maturity, innovate for growth

Write the code to display a given plot [closed]

2 Answers

Self-driving car companies complain California test data may mislead

Google launches TensorFlow library for optimizing fairness constraints

Researchers propose data set to measure few-shot learning performance

Recent Posts

Categories

Archives