researchers Archives - Business Intelligence Info

Tag Archives: researchers

Robotics researchers propose AI that locates and safely moves items on shelves

November 29, 2020 Big Data

A pair of new robotics studies from Google and the University of California, Berkeley propose ways of finding occluded objects on shelves and solving “contact-rich” manipulation tasks like moving objects across a table. The UC Berkeley research introduces Lateral Access maXimal Reduction of occupancY support Area (LAX-RAY), a system that predicts a target object’s location, even when only a portion of that object is visible. As for the Google-coauthored paper, it proposes Contact-aware Online COntext Inference (COCOI), which aims to embed the dynamics properties of physical things in an easy-to-use framework.

While researchers have explored the robotics problem of searching for objects in clutter for quite some time, settings like shelves, cabinets, and closets are a less-studied area, despite their wide applicability. (For example, a service robot at a pharmacy might need to find supplies from a medical cabinet.) Contact-rich manipulation problems are just as ubiquitous in the physical world, and humans have developed the ability to manipulate objects of various shapes and properties in complex environments. But robots struggle with these tasks due to the challenges inherent in comprehending high-dimensional perception and physics.

The UC Berkeley researchers, working out of the university’s AUTOLab department, focused on the challenge of finding occluded target objects in “lateral access environments,” or shelves. The LAX-RAY system comprises three lateral access mechanical search policies. Called “Uniform,” “Distribution Area Reduction (DAR),” and “Distribution Area Reduction over ‘n’ steps (DER-n),” they compute actions to reveal occluded target objects stored on shelves. To test the performance of these policies, the coauthors leveraged an open framework — The First Order Shelf Simulator (FOSS) — to generate 800 random shelf environments of varying difficulty. Then they deployed LAX-RAY to a physical shelf with a Fetch robot and an embedded depth-sensing camera, measuring whether the policies could figure out the locations of objects accurately enough to have the robot push those objects.

The researchers say the DAR and DER-n policies showed strong performance compared with the Uniform policy. In a simulation, LAX-RAY achieved 87.3% accuracy, which translated to about 80% accuracy when applied to the real-world robot. In future work, the researchers plan to investigate more sophisticated depth models and the use of pushes parallel to the camera to create space for lateral pushes. They also hope to design pull actions using pneumatically activated suction cups to lift and remove occluding objects from crowded shelves.

In the Google work, which had contributions from researchers at Alphabet’s X, Stanford, and UC Berkeley, the coauthors designed a deep reinforcement learning method that takes multimodal data and uses a “deep representative structure” to capture contact-rich dynamics. COCOI taps video footage and readings from a robot-mounted touch sensor to encode dynamics information into a representation. This allows a reinforcement learning algorithm to plan with “dynamics-awareness” that improves its robustness in difficult environments.

The researchers benchmarked COCOI by having both a simulated and real-world robot push objects to target locations while avoiding knocking them over. This isn’t as easy as it sounds; key information couldn’t be easily extracted from third-angle perspectives, and the task dynamics properties weren’t directly observable from raw sensor information. Moreover, the policy needed to be effective for objects with different appearances, shapes, masses, and friction properties.

The researchers say COCOI outperformed a baseline “in a wide range of settings” and dynamics properties. Eventually, they intend to extend their approach to pushing non-rigid objects, such as pieces of cloth.

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers investigate why popular AI algorithms classify objects by texture, not by shape

November 15, 2020 Big Data

In a paper accepted to the 2020 NeurIPS conference, Google and Stanford researchers explore the bias exhibited by certain kinds of computer vision algorithms — convolutional neural networks (CNNs) — trained on the open source ImageNet dataset. Unlike humans, ImageNet-trained CNNs tend to classify images by texture rather than by shape. Their work indicates that CNNs’ bias toward textures may arise not from differences in their internal workings but from differences in the data that they see.

CNNs attain state-of the-art results in computer vision tasks including image classification, object detection, and segmentation. Although their performance in several of these tasks approaches that of humans, recent findings show that CNNs differ in key ways from human vision. For example, recent work compared humans to ImageNet-trained CNNs on a dataset of images with conflicting shape and texture information (e.g. an elephant-textured knife), concluding that models tend to classify according to material (e.g. “checkered”) and humans to shape (e.g. “circle”).

The Google and Stanford team discovered that “naturalistic” data augmentation involving color distortion, noise, and blur can decrease this CNN texture bias, whereas “random-crop” augmentation increases the bias. Combining these observations, they trained models that classify ambiguous images by shape a majority of the time. These models also ostensibly outperform baselines on datasets that exemplify different notions of shape.

CNN model architectures that perform better on ImageNet generally have less texture bias, according to the researchers, but architectures designed to match the human visual system don’t have biases substantially different from ordinary CNNs. In the course of experimentation, the researchers also discovered that it’s possible to extract more shape information from a CNN than is reflected in the model’s classifications.

As the coauthors note, people who build and interact with tools for computer vision — especially those without extensive training in machine learning — often have a mental model of computer vision models that’s similar to human vision. But the paper’s findings build on a body of work showing this view is incorrect. Differences between human and machine vision of the kind the coauthors studied could cause data scientists to make significant errors in anticipating and reasoning about the behavior of computer vision systems. They advocate allowing people from a range of backgrounds to make safe, predictable, and equitable models requiring vision systems to perform at least roughly in accordance with their expectations.

“Making computer vision models that share the same inductive biases as humans is an important step towards this goal,” the researchers wrote. “At the same time, we recognize the possible negative consequences of blindly constraining models’ judgments to agree with people’s: human visual judgments display forms of bias that should be kept out of computer models. More broadly, we believe that work like ours can have a beneficial impact on the internal sociology of the machine learning community. By identifying connections to developmental psychology and neuroscience, we hope to enhance interdisciplinary connections across fields, and to encourage people with a broader range of training and backgrounds to participate in machine learning research.”

Best practices for a successful AI Center of Excellence:

A guide for both CoEs and business units Access here

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers find flaws in algorithm used to identify atypical medication orders

November 5, 2020 Big Data

Can algorithms identify unusual medication orders or profiles more accurately than humans? Not necessarily. A study coauthored by researchers at the Université Laval and CHU Sainte-Justine in Montreal found that one model used by physicians to screen patients performed poorly on some orders. It’s a cautionary tale of the use of AI and machine learning in medicine, where unvetted technology has the potential to negatively impact outcomes.

Pharmacists review lists of active medications — i.e., pharmacological profiles — for inpatients under their care. This process aims to identify medications that could be abused, but most medication orders don’t show drug-related problems. Publications from over a decade ago illustrate the potential of technology to help pharmacists streamline workflows like order review, but while more recent research has investigated AI’s potential in pharmacology, few studies have demonstrated efficacy.

The coauthors of this latest work looked at a model deployed in a tertiary-care mother-and-child academic hospital between April 2020 and August 2020. The model was trained on a dataset of 2,846,502 medication orders from 2005 to 2018 extracted from a pharmacy database and preprocessed into 1,063,173 profiles. Prior to data collection, every month, the model was retrained with ten years of most recent data from the database in order to minimize drift, which occurs when a model loses its predictive power.

Pharmacists at the academic hospital rated medication order in the database as “typical” or “atypical” before observing the predictions; patients were evaluated only once to minimize the risk of including profiles that the pharmacists had previously evaluated. Atypical prescriptions were defined as those that didn’t correspond to the usual prescribing patterns, according to the pharmacist’s expertise, while profiles were considered atypical if at least one medication order within them was labeled as atypical.

The model’s profile predictions were provided to the pharmacists and they indicated whether they agreed or disagreed with each prediction. In all, 12,471 medication orders and 1,356 profiles were shown to 25 pharmacists from seven of the academic hospital’s departments, mostly from obstetrics-gynecology.

The researchers report that the model exhibited poor performance with respect to medication orders, attaining an F1-score of 0.30 (lower is worse). On the other hand, the model’s profile predictions achieved “satisfactory” performance with an F1-score of 0.59.

One reason might be a lack of representative data; research has shown that biased diagnostic algorithms may perpetuate inequalities. A team of scientists recently found that almost all eye disease datasets come from patients in North America, Europe, and China, meaning eye disease-diagnosing algorithms are less certain to work well for racial groups from underrepresented countries. In another study, Stanford University researchers claimed that most of the U.S. data for studies involving medical uses of AI come from California, New York, and Massachusetts.

Cognizant of this, the coauthors of this study say they don’t believe the model could be used as a standalone decision support tool. However, they believe it could be combined with rules-based approaches to identify medication order issues independent of common practice. “Conceptually, presenting pharmacists with a prediction for each order should be better because it identifies clearly which prescription is atypical, unlike profile predictions which only inform the pharmacist that something is atypical within the profile,” they wrote. “Although [our] focus groups indicated a lack of trust in order predictions by pharmacists, they were satisfied to use them as a safeguard to ensure that they did not miss unusual orders. This leads us to believe that even moderately improving the quality of these predictions in future work could be beneficial.”

How startups are scaling communication:

The pandemic is making startups take a close look at ramping up their communication solutions. Learn how

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers develop AI that solves a matrix-based visual cognitive test

November 4, 2020 Big Data

Multiple choice tests provide test-takers the ability to compare answers to eliminate choices (or guess the correct one). Each choice can be compared with the question to infer patterns that might have been missed; it’s arguably the ability to narrow down the right answer from sets of answers that’s the test of true comprehension.

Inspired by this, researchers at Tel Aviv University and Facebook developed a machine learning model that generates answers to the Raven Progressive Matrix (RPM), a type of intelligence test where the goal is to complete the location in a grid of abstract images. The coauthors claim that their algorithm is not only able to generate a plausible set of answers competitive with state-of-the-art methods, but that it could be used to build an automatic tutoring system that adjusts to the proficiencies of individual students.

RPM is a nonverbal test typically used in educational settings like schools. It’s usually a 60-item exam given to measure abstract reasoning, which is regarded as a nonverbal estimate of fluid intelligence (i.e., the ability to solve novel reasoning problems). Each question — a single problemz– consists of eight images placed
on a grid of size 3 x 3. The task is to generate the missing ninth image on the third row of the third column such that it matches the patterns of the rows and columns of the grid.

RPM combines what the researchers describe as pathways: reconstruction, recognition, and generation. The reconstruction pathway provides supervision so that each image is encoded into a numerical representation and aggregated along rows and columns. The recognition pathway shapes the representations in a way that makes the semantic information more explicit. As for the generation pathway, it relies on embedding the visual representation from the first pathway and the semantic embedding obtained with the assistance of the second to map the semantic representation of a given question to an image.

In an experiment involving a dataset of matrices problems called RAVEN-FAIR, the researchers report that their model attained 60.8% accuracy overall. “Our method presents very convincing generation results. The state of the art recognition methods regard the generated answer as the right one in a probability that approaches that of the ground truth answer,” they wrote. “This is despite the non-deterministic nature of the problem, which means that the generated answer is often completely different … from the ground truth image. In addition, we demonstrate that the generation capability captures most rules, with little neglect of specific ones.”

Beyond potential applications in education, the researchers assert that the shift from selecting an answer from a closed set to generating an answer could lead to more interpretable machine learning methods. Because the generated output may reveal information about the underlying inference process, models like theirs, they say, could be useful in validating machine logic through the implementation of AI systems.

How startups are scaling communication:

The pandemic is making startups take a close look at ramping up their communication solutions. Learn how

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers design virtual environment to spur development of helpful home robots

October 25, 2020 Big Data

Without much prior experience, kids can recognize other people’s intentions and come up with plans to help them achieve their goals, even in novel scenarios. By contrast, even the most sophisticated AI systems to date still struggle with basic social interactions. That’s why researchers at MIT, Nvidia, and ETH Zurich developed Watch-And-Help (WAH), a challenge in which embodied AI agents need to understand goals by watching a demonstration of a human performing a task and coordinating with the human to solve the task as quickly as possible.

The concept of embodied AI draws on embodied cognition, the theory that many features of psychology — human or otherwise — are shaped by aspects of the entire body of an organism. By applying this logic to AI, researchers hope to improve the performance of AI systems like chatbots, robots, autonomous vehicles, and even smart speakers that interact with their environments, people, and other AI. A truly embodied robot could check to see whether a door is locked, for instance, or retrieve a smartphone that’s ringing in an upstairs bedroom.

In the first phase of WAH, which the researchers call the Watch stage, an AI agent observes a humanlike agent perform a task and infers a goal from their actions. In the second stage — the Help stage — the AI agent assists the humanlike agent in achieving the same goal in a completely different environment. The researchers assert that this two-stage framework poses unique challenges for human-AI collaboration because the AI agent has to reason about the humanlike agent’s intention and generalize its knowledge about the goal.

To enable the kinds of interactions involved in WAH, the researchers had to extend the open source platform VirtualHome and build a multi-agent environment dubbed VirtualHome-Social. VirtualHome-Social simulates home settings so agents can interact with different objects and agents, for example opening a container or grabbing a utensil from a drawer. VirtualHome-Social also provides built-in agents that emulate human behaviors and an interface for human players. This enables testing with real humans and human activities displayed in semi-realistic environments.

The humanlike agent represents a built-in agent in VirtualHome-Social. It plans its actions based on a goal and its observation of the environment. During the Help stage, the AI agent receives observations from the system at each step and sends an action command back to control a virtual avatar. Meanwhile, the humanlike agent — which can also be controlled by a human — updates its plan based on its latest observation to reflect any state change caused by the AI agent.

The researchers designed an evaluation protocol and provided benchmarks for WAH, including a goal model for the Watch stage and multiple planning and machine learning baselines for the Help stage. The team says results indicate that to achieve success in WAH, AI agents must acquire strong social perception and generalizable helping strategies — as hypothesized.

“Our ultimate goal is to build AI agents that can work with real humans. Our platform opens up exciting directions of future work, such as online goal inference and direct communication between agents,” the researchers wrote. “We hope that the proposed challenge and virtual environment can promote future research on building more sophisticated machine social intelligence.”

The audio problem:

Learn how new cloud-based API solutions are solving imperfect, frustrating audio in video conferences. Access here

Let’s block ads! (Why?)

Big Data – VentureBeat

AI researchers urge tech to go beyond scale to address systemic social issues

October 24, 2020 Big Data

The definition of success for startups and Big Tech companies alike has long been summed up by three words: hockey stick growth. Speedy gains in terms of both users and revenue is the dream for any company looking to scale. But according to a paper recently published by Google senior research scientist Alex Hanna and independent researcher Tina Park, a growing number of AI researchers say companies interested in purpose beyond profit need to consider approaches beyond rapid growth.

The paper argues that scale thinking is not just a way to grow a business, but a method that impacts all parts of that business, actively inhibits participation in tech and society, and “forces particular types of participation to operate as extractive or exploitative labor.”

“Whether people are aware of it or not, scale thinking is all-encompassing. It is not just an attribute of one’s product, service, or company, but frames how one thinks about the world (what constitutes it and how it can be observed and measured), its problems (what is a problem worth solving versus not), and the possible technological fixes for those problems,” the paper reads.

The authors go on to say that companies rooted in scale thinking are unlikely to be as “effective at deep, systemic change as their purveyors imagine. Rather, solutions which resist scale thinking are necessary to undo the social structures which lie at the heart of social inequality.”

This kind of thinking runs counter to not only dogma at the heart of Big Tech companies like Facebook and Google, but also the way media and analysts typically assess the value of emerging startups.

Earlier this month, Congress released an antitrust investigation that found Big Tech companies rely on scale to maintain and strengthen monopolies across the digital economy. A Department of Justice (DOJ) lawsuit filed Tuesday against Google, the first against a major tech company in two decades, also points to the scale achieved through algorithms and collection of personal user data as a key factor in the government’s decision to sue the Alphabet company.

The opposition includes scale evangelists like Y Combinator cofounder Paul Graham, AWS CTO Werner Vogels, and former Google CEO Eric Schmidt, who is quoted in the DOJ lawsuit as saying “scale is the key” to Google’s strength in search.

Embedded in scale thinking, Hanna and Park argue, is the idea that scalability is morally good and solutions that cannot scale are morally impoverished. Authors say that’s part of why Big Tech companies place such a high value on artificial intelligence.

“Large tech firms spend much of their time hiring developers who can envision solutions which can be implemented algorithmically. Code and algorithms which scale poorly are seen as undesirable and inefficient. Many of the most groundbreaking infrastructural developments in Big Tech have been those which increase scalability, such as Google File System (and subsequently the MapReduce computing schema) and distributed and federated machine learning models,” the paper reads.

Hanna and Park also characterize scale thinking as shortsighted because it requires companies to treat resources and people as interchangeable units and encourages the datafication of users in order to “find ways to rationalize the individual into legible data points.” This approach can lead to systems that are not made to serve everyone equally and that negatively impact the lives of those who fall outside their scaled solutions.

The paper also notes that scale thinking is an inefficient way to increase hiring or retention of employees from diverse backgrounds. Since the deaths of Black Americans like Breonna Taylor and George Floyd led to calls for racial justice earlier this year, a number of major tech companies have recommitted to diversity goals, but for years now progress has been virtually undetectable. Examples in the paper include a tendency to focus on bias workshops or other inclusion metrics rather than the experiences of marginalized people within the company.

Rather than making scale a company’s North Star, the authors suggest approaches like “mutual aid,” in which businesses adopt an interdependent model and take responsibility for meeting the direct material needs of individuals. The idea of mutual aid arose in part from the kinds of support systems that sprung up in the wake of the COVID-19 pandemic.

“While scale thinking emphasizes abstraction and modularity, mutual aid networks encourage concretization and connection,” the paper reads. “While mutual aid is not the only framework through which we can consider a move away from scale thinking-based collaborative work arrangements, we find it to be a fruitful one to theorize and pursue.”

In addition to exploring mutual aid, the paper encourages developers to ask questions about any system they create, such as whether it legitimizes or expands social systems people are trying to dismantle, whether it encourages broad participation, and whether it centralizes power or distributes power among developers and users.

Recommendations are in line with a range of ethically centered technology models proposed by members of the AI community in recent months. Other approaches include the idea of anticolonial AI, which rejects algorithmic oppression and data colonization, queering machine learning, data feminism, and building AI based on the African philosophy of Ubuntu, which focuses on the interconnectedness of people and the natural world.

There’s also “Good Intentions, Bad Inventions,” a Data & Society primer published earlier this month that attempts to dispel common myths about the best ways to build technology and improve user well-being.

Titled “Against Scale: Provocations and Resistances to Scale Thinking,” the paper was highlighted this week at a Computer-Supported Cooperative Work and Social Computing (CSCW) conference workshop. Before writing critically about scale, Hanna and colleagues at Google published a paper in late 2019 that argues the algorithmic fairness community should look to critical race theory as a way to interrogate AI systems and their impact on human lives.

The audio problem:

Learn how new cloud-based API solutions are solving imperfect, frustrating audio in video conferences. Access here

Let’s block ads! (Why?)

Big Data – VentureBeat

arXiv now allows researchers to submit code with their manuscripts

October 11, 2020 Big Data

Papers with Code today announced that preprint paper archive arXiv will now allow researchers to submit code alongside research papers, giving computer scientists an easy way to analyze, scrutinize, or reproduce claims of state-of-the-art AI or novel advances in what’s possible.

An assessment of the AI industry released a week ago found that only 15% of papers submitted by researchers today publish their code.

Maintained by Cornell University, arXiv hosts manuscripts from fields like biology, mathematics, and physics, and it has become one of the most popular places online for artificial intelligence researchers to publicly share their work. Preprint repositories give researchers a way to share their work immediately, before undergoing what can be a long peer review process as practiced by reputable scholarly journals. Code shared on arXiv will be submitted through Papers with Code and can be found in a Code tab for each paper.

“Having code on arXiv makes it much easier for researchers and practitioners to build on the latest machine learning research,” Papers with Code cocreator Robert Stojnic said a blog post today. “We also hope this change has ripple effects on broader computational science beyond machine learning. Science is cumulative. Open science, including making available key artefacts such as code, helps to accelerate progress by making research easier to build upon.”

Started in 2018, Papers with Code focuses on encouraging reproducibility of AI model results and, as the name states, submitting research with code. The Papers with Code website shares nearly 2,000 papers and code from across major fields in AI like natural language processing, computer vision, adversarial machine learning, and robotics. Papers with Code was initially founded in part by members of Facebook AI Research. Last year, Facebook and Papers with Code launched PyTorch Hub to encourage reproducibility.

In the past year or so, sharing code along with a research paper manuscript has become standard at major AI research conferences. At ICML 2019, nearly 70% of authors submitted code with their papers by the start of the conference. ICML organizers found that 90% of researchers who submitted code came from academia, and about 27% included an author from industry. Conversely, nearly 84% of authors overall came from industry and about 27% from academia. People developing software or AI inside companies may be more likely to view secrecy as important to protecting intellectual property or financial interests.

NeurIPS began to experiment with a code submission policy in 2019 and put an official code submission policy into effect this year. In other news about evolving standards for AI researchers, earlier this year NeurIPS began to require all paper submissions to include social impact statements and declare any potential financial conflicts of interest.

Let’s block ads! (Why?)

Big Data – VentureBeat

Allen Institute researchers find pervasive toxicity in popular language models

September 27, 2020 Big Data

Researchers at the Allen Institute for AI have created a data set — RealToxicityPrompts — that attempts to elicit racist, sexist, or otherwise toxic responses from AI language models, as a way of measuring the models’ preferences for these responses. In experiments, they claim to have found that no current machine learning technique sufficiently protects against toxic outputs, underlining the need for better training sets and model architectures.

It’s well-established that models amplify the biases in data on which they were trained. That’s problematic in the language domain, because a portion of the data is often sourced from communities with pervasive gender, race, and religious prejudices. AI research firm OpenAI notes that this can lead to placing words like “naughty” or “sucked” near female pronouns and “Islam” near words like “terrorism.” Other studies, like one published by Intel, MIT, and Canadian AI initiative CIFAR researchers in April, have found high levels of stereotypical bias from some of the most popular models, including Google’s BERT and XLNet, OpenAI’s GPT-2, and Facebook’s RoBERTa.

The Allen Institute researchers designed RealToxicityPrompts to measure the risk of “toxic degeneration” by pretrained language models, or models fed data sets containing thousands to billions of documents. They compiled a list of 100,000 naturally occurring prompts extracted from a large corpus of English Reddit text (the open source Open-WebText Corpus) and paired it with toxicity scores from Google’s Perspective API, which uses machine learning models to detect the potential toxicity of a comment.

The coauthors evaluated five language models using RealToxicityPrompts, specifically three models from OpenAI (GPT-1 GPT-2, and GPT-3) and two models from Salesforce (CTRL and CTRL-Wiki). The found that while toxic prompts — prompts offensive or stereotypically biased on their face — were 70% or more likely to yield toxic content from the language models, even non-toxic prompts resulted in offensive responses. The results show that all models were 49% or more likely to answer non-toxic content with toxic responses, even models like CTRL-Wiki that were only trained on Wikipedia data.

To uncover the potential reasons for this, the researchers investigated the corpora used to pretrain several of the language models: OpenAI-WT (GPT-2’s training data) and OWTC (an open source fork of OpenAI-WT). OWTC contains text from Reddit posts with a karma of 3 or higher and 38GB of English documents, including news articles. OpenAI-WT — which has a 29% overlap with OWTC, such that at least 2.3 million documents in OpenAI-WT also appear in OWTC — contains about 8 million documents filtered using a blocklist of sexually explicit and otherwise offensive subreddits.

The researchers found that OWTC and OpenAI-WT contain “non-negligible” amounts of toxicity as identified by the Perspective API. About 2.1% of documents in OWTC were offensive compared with 4.3% in OpenAI-WT, or twice that of OWTC despite the blocklist. Unreliable news sites were another major source of toxicity in the data sets, as were posts from banned or quarantined subreddits. In fact, 63,000 documents in OpenAI-WT and OWTC came from links shared on problematic Reddit communities; GPT-2 was pretrained on at least 40,000 documents from the quarantined /r/The_Donald and 4,000 documents from the banned /r/WhiteRights.

“Overall, our investigations demonstrate that toxicity is a prevalent issue in both neural language generation and web text corpora,” the coauthors wrote in a paper describing their work. “Although they show some reduction in toxicity, steering methods do not fully protect neural models from toxic degeneration. Additionally, the corpora that language models are pretrained on contain non-negligible amounts of toxic, abusive, and untrustworthy content.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers create dataset to advance U.S. Supreme Court gender bias analysis

September 22, 2020 Big Data

University of Washington language researchers and legal professionals recently created a labeled dataset for detection of interruptions and competitive turn-taking in U.S. Supreme Court oral arguments. They then used the corpus of “turn changes” to train AI models to experiment with ways to automatically classify turn changes as competitive or cooperative as a way to analyze gender bias.

“In-depth studies of gender bias and inequality are critical to the oversight of an institution as influential as the Supreme Court,” reads the paper University of Washington researchers Haley Lepp and Gina-Anne Levow published on preprint repository arXiv one week ago. “We find that as the first person in an exchange, female speakers and attorneys are spoken to more competitively than are male speakers and justices. We also find that female speakers and attorneys speak more cooperatively as the second person in an exchange than do male speakers and justices.”

Attorneys who speak before the Supreme Court are allotted 30 minutes of oral argument and are expected to stop talking when a justice speaks. Linguists have observed men interrupting women routinely in professional environments and other settings.

Turn changes are defined as instances when one person stops speaking and another person starts speaking. Short audio clips of each turn change were annotated as competitive or cooperative by 77 members of the U.S. legal community who identify as an attorney, judge, legal scholar, or law student in their second year or higher. Lepp and Levow’s work focuses on measuring whether the turn change was cooperative or competitive, based on oral argument audio the Supreme Court made available, in part because previous work by Deborah Tannen found that interruptions in speech can be part of regular discourse and that the context of the conversation can be a factor.

The paper devoted to gender bias analysis was published days before the death of Supreme Court Justice Ruth Bader Ginsburg at the age of 87. Ginsburg was the second woman ever appointed to the U.S. Supreme Court. As a litigator for the American Civil Liberties Union (ACLU), Ginsburg successfully argued cases before the Supreme Court that greatly extended women’s rights in the United States. On Wednesday and Thursday, she will be the first woman and the first Jewish person in U.S. history to lie in state at the U.S. Capital building for members of the public to say goodbye. She was the longest-serving female justice in U.S. history.

Although voting has already begun in some parts of the country and Ginsburg pleaded in her final days to let the winner of the presidential election fill her vacancy, President Trump is expected to nominate a pick to fill her seat Friday or Saturday. Two Republican Senators pledged not to vote until the presidential election is decided, but Senate Majority Leader Mitch McConnell said just hours after her death that the president’s nominee will get a vote.

Details of the turn changes corpus dataset follow a 2017 study that used automation to identify the number of interruptions that occurred from 2004-2015. The study “Justice, Interrupted: The Effect of Gender, Ideology and Seniority at Supreme Court Oral Arguments” by Tonja Jacobi and Dylan Schweers found that women are interrupted three times as often as male Supreme Court justices are. Female Supreme Court justices were interrupted by attorneys as well as other Supreme Court justices, led by Anthony Kennedy, Antonin Scalia, and William Rehnquist. Scalia and Stephen Breyer also interrupted each other a lot.

A producer of the podcast More Perfect noticed people repeatedly interrupting Ginsburg, which led to an episode on the subject. Jacobi spoke on the podcast and said Ginsburg developed tactics to adapt to frequent interruptions, first by asking to ask a question, then pivoting to ask questions more like male justices who interrupt.

The episode also highlighted that Justice Sonia Sotomayor was found to speak as often as men in the Jacobi study, but has still drawn criticism from media commentators at times for being aggressive. Gender is pervasive in coverage of Supreme Courts, according to a 2016 analysis of media coverage in five democratic countries. The analysis found that generally women who ask questions like male justices are labeled abrasive, militant, or mean by critics.

Last year, the U.S. Supreme Court introduced a rule that justices will try to give attorneys two minutes to speak without interruption at the start of oral arguments.

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers claim masks muffle speech, but not enough to impede speech recognition

September 12, 2020 Big Data

Health organizations including the U.S. Centers for Disease Control and Prevention, the World Health Organization, and the U.K. National Health Service advocate wearing masks to prevent the spread of infection. But masks attenuate speech, which has implications for the accuracy of speech recognition systems like Google Assistant, Alexa, and Siri. In an effort to quantify the degree to which mask materials impact acoustics, researchers at the University of Illinois conducted a study examining 12 different types of face coverings in total. They found that transparent masks had the worst acoustics compared with both medical and cloth masks, but that most masks had “little effect” on lapel microphones, suggesting existing systems might be able to recognize muffled speech without issue.

While it’s intuitive to assume mask-distorted speech would prove to be challenging for speech recognition, the evidence so far paints a mixed picture. Research published by the Educational Testing Service (ETS) concluded that while differences existed between recordings of mask wearers and those who didn’t wear masks during an English proficiency exam, the distortion didn’t lead to “significant” variations in automated exam scoring. In a separate study, scientists at Duke Kunshan University, Lenovo, and Wuhan University found an AI system could be trained to detect whether someone’s wearing a mask from the sound of their muffled speech.

A Google spokesperson told VentureBeat there hasn’t been a measurable impact on the company’s speech recognition systems since the start of the pandemic, when mask-wearing became more common. Amazon also says it hasn’t observed a shift in speech recognition accuracy correlated with mask-wearing.

The University of Illinois researchers looked at the acoustic effects of a polypropylene surgical mask, N95 and KN95 respirators, six cloth masks made from different fabrics, two cloth masks with transparent windows, and a plastic shield. They took measurements within an “acoustically-treated” lab using a head-shaped loudspeaker and a human volunteer, both of whom had microphones placed on and near their lapel, cheek, forehead, and mouth. (The head-shaped loudspeaker, which was made of plywood, used a two-inch driver with a pattern close to that of a human speaker.)

After taking measurements without face coverings to establish a baseline, the researchers set the loudspeaker on a turntable and rotated it to capture various angles of the tested masks. Then, for each mask, they had the volunteer speak in three 30-second increments at a constant volume.

The results show that most masks had “little effect” below a frequency of 1kHz but were muffled at higher frequencies in varying degrees. The surgical mask and KN95 respirator had peak attenuation of around 4dB, while the N95 attenuated at high frequencies by about 6dB. As for the cloth masks, material and weave proved to be the key variables — 100% cotton masks had the best acoustic performance, while masks made from tightly woven denim and bedsheets performed the worst. Transparent masks blocked between 8dB and 14dB at high frequencies, making them by far the worst of the bunch.

“For all masks tested, acoustic attenuation was strongest in the front. Sound transmission to the side of and behind the talker was less strongly affected by the masks, and the shield amplified sound behind the talker,” the researchers in a paper describing their work. “These results suggest that masks may deflect sound energy to the sides rather than absorbing it. Therefore, it may be possible to use microphones placed to the side of the mask for sound reinforcement.”

The researchers recommend avoiding cotton-spandex masks for the clearest and crispest speech, but they note that recordings captured by the lapel mic showed “small” and “uniform” attenuation — the sort of attenuation that recognition systems can easily correct for. For instance, Amazon recently launched Whisper Mode for Alexa, which taps AI trained on a corpus of professional voice recordings to respond to whispered (i.e., low-decibel) speech by whispering back. An Amazon spokesperson didn’t say whether Whisper Mode is being used to improve masked speech performance, but they told VentureBeat that when Alexa speech recognition systems’ signal-to-noise ratios are lower due to customers wearing masks, engineering teams are able to address fluctuations in confidence through an active learning pipeline.

In any case, assuming the results of the University of Illinois stand up to peer review, they bode well for smart speakers, smart displays, and other voice-powered smart devices. Next time you lift your phone to summon Siri, you shouldn’t have to ditch the mask.

Let’s block ads! (Why?)

Big Data – VentureBeat

Tag Archives: researchers

Robotics researchers propose AI that locates and safely moves items on shelves

Researchers investigate why popular AI algorithms classify objects by texture, not by shape

Researchers find flaws in algorithm used to identify atypical medication orders

Researchers develop AI that solves a matrix-based visual cognitive test

Researchers design virtual environment to spur development of helpful home robots

arXiv now allows researchers to submit code with their manuscripts

Allen Institute researchers find pervasive toxicity in popular language models

Researchers create dataset to advance U.S. Supreme Court gender bias analysis

Researchers claim masks muffle speech, but not enough to impede speech recognition

Recent Posts

Categories

Archives