language Archives - Business Intelligence Info

Tag Archives: language

Allen Institute researchers find pervasive toxicity in popular language models

September 27, 2020 Big Data

Researchers at the Allen Institute for AI have created a data set — RealToxicityPrompts — that attempts to elicit racist, sexist, or otherwise toxic responses from AI language models, as a way of measuring the models’ preferences for these responses. In experiments, they claim to have found that no current machine learning technique sufficiently protects against toxic outputs, underlining the need for better training sets and model architectures.

It’s well-established that models amplify the biases in data on which they were trained. That’s problematic in the language domain, because a portion of the data is often sourced from communities with pervasive gender, race, and religious prejudices. AI research firm OpenAI notes that this can lead to placing words like “naughty” or “sucked” near female pronouns and “Islam” near words like “terrorism.” Other studies, like one published by Intel, MIT, and Canadian AI initiative CIFAR researchers in April, have found high levels of stereotypical bias from some of the most popular models, including Google’s BERT and XLNet, OpenAI’s GPT-2, and Facebook’s RoBERTa.

The Allen Institute researchers designed RealToxicityPrompts to measure the risk of “toxic degeneration” by pretrained language models, or models fed data sets containing thousands to billions of documents. They compiled a list of 100,000 naturally occurring prompts extracted from a large corpus of English Reddit text (the open source Open-WebText Corpus) and paired it with toxicity scores from Google’s Perspective API, which uses machine learning models to detect the potential toxicity of a comment.

The coauthors evaluated five language models using RealToxicityPrompts, specifically three models from OpenAI (GPT-1 GPT-2, and GPT-3) and two models from Salesforce (CTRL and CTRL-Wiki). The found that while toxic prompts — prompts offensive or stereotypically biased on their face — were 70% or more likely to yield toxic content from the language models, even non-toxic prompts resulted in offensive responses. The results show that all models were 49% or more likely to answer non-toxic content with toxic responses, even models like CTRL-Wiki that were only trained on Wikipedia data.

To uncover the potential reasons for this, the researchers investigated the corpora used to pretrain several of the language models: OpenAI-WT (GPT-2’s training data) and OWTC (an open source fork of OpenAI-WT). OWTC contains text from Reddit posts with a karma of 3 or higher and 38GB of English documents, including news articles. OpenAI-WT — which has a 29% overlap with OWTC, such that at least 2.3 million documents in OpenAI-WT also appear in OWTC — contains about 8 million documents filtered using a blocklist of sexually explicit and otherwise offensive subreddits.

The researchers found that OWTC and OpenAI-WT contain “non-negligible” amounts of toxicity as identified by the Perspective API. About 2.1% of documents in OWTC were offensive compared with 4.3% in OpenAI-WT, or twice that of OWTC despite the blocklist. Unreliable news sites were another major source of toxicity in the data sets, as were posts from banned or quarantined subreddits. In fact, 63,000 documents in OpenAI-WT and OWTC came from links shared on problematic Reddit communities; GPT-2 was pretrained on at least 40,000 documents from the quarantined /r/The_Donald and 4,000 documents from the banned /r/WhiteRights.

“Overall, our investigations demonstrate that toxicity is a prevalent issue in both neural language generation and web text corpora,” the coauthors wrote in a paper describing their work. “Although they show some reduction in toxicity, steering methods do not fully protect neural models from toxic degeneration. Additionally, the corpora that language models are pretrained on contain non-negligible amounts of toxic, abusive, and untrustworthy content.”

Let’s block ads! (Why?)

Big Data – VentureBeat

AI Weekly: Cutting-edge language models can produce convincing misinformation if we don’t stop them

September 19, 2020 Big Data

It’s been three months since OpenAI launched an API underpinned by cutting-edge language model GPT-3, and it continues to be the subject of fascination within the AI community and beyond. Portland State University computer science professor Melanie Mitchell found evidence that GPT-3 can make primitive analogies, and Columbia University’s Raphaël Millière asked GPT-3 to compose a response to the philosophical essays written about it. But as the U.S. presidential election nears, there’s growing concern among academics that tools like GPT-3 could be co-opted by malicious actors to foment discord by spreading misinformation, disinformation, and outright lies. In a paper published by the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism (CTEC), the coauthors find that GPT-3’s strength in generating “informational,” “influential” text could be leveraged to “radicalize individuals into violent far-right extremist ideologies and behaviors.”

Bots are increasingly being used around the world to sow the seeds of unrest, either through the spread of misinformation or the amplification of controversial points of view. An Oxford Internet Institute report published in 2019 found evidence of bots disseminating propaganda in 50 countries, including Cuba, Egypt, India, Iran, Italy, South Korea, and Vietnam. In the U.K., researchers estimate that half a million tweets about the country’s proposal to leave the European Union sent between June 5 and June 12 came from bots. And in the Middle East, bots generated thousands of tweets in support of Saudi Arabia’s crown prince Mohammed bin Salman following the 2018 murder of Washington Post opinion columnist Jamal Khashoggi.

Bot activity perhaps most relevant to the upcoming U.S. elections occurred last November, when cyborg bots spread misinformation during the local Kentucky elections. VineSight, a company that tracks social media misinformation, uncovered small networks of bots retweeting and liking messages casting doubt on the gubernatorial results before and after the polls closed.

But bots historically haven’t been sophisticated; most simply retweet, upvote, or favorite posts likely to prompt toxic (or violent) debate. GPT-3-powered bots or “cyborgs” — accounts that attempt to evade spam detection tools by fielding tweets from human operators — could prove to be far more harmful given how convincing their output tends to be. “Producing ideologically consistent fake text no longer requires a large corpus of source materials and hours of [training]. It is as simple as prompting GPT-3; the model will pick up on the patterns and intent without any other training,” the coauthors of the Middlebury Institute study wrote. “This is … exacerbated by GPT-3’s impressively deep knowledge of extremist communities, from QAnon to the Atomwaffen Division to the Wagner Group, and those communities’ particular nuances and quirks.”

AI Weekly: Cutting edge language models can produce convincing misinformation if we don’t stop them

Above: A question-answer thread generated by GPT-3.

In their study, the CTEC researchers sought to determine whether people could color GPT-3’s knowledge with ideological bias. (GPT-3 was trained on trillions of words from the internet, and its architectural design enables fine-tuning through longer, representative prompts like tweets, paragraphs, forum threads, and emails.) They discovered that it only took a few seconds to produce a system able to answer questions about the world consistent with a conspiracy theory, in one case falsehoods originating from the QAnon and Iron March communities.

“GPT-3 can complete a single post with convincing responses from multiple viewpoints, bringing in various different themes and philosophical threads within far-right extremism,” the coauthors wrote. “It can also generate new topics and opening posts from scratch, all of which fall within the bounds of [the communities’] ideologies.”

CTEC’s analysis also found GPT-3 is “surprisingly robust” with respect to multilingual language understanding, demonstrating an aptitude for producing Russian-language text in response to English prompts that show examples of right-wing bias, xenophobia, and conspiracism. The model also proved “highly effective” at creating extremist manifestos that were coherent, understandable, and ideologically consistent, communicating how to justify violence and instructing on anything from weapons creation to philosophical radicalization.

Above: GPT-3 writing extremist manifestos.

“No specialized technical knowledge is required to enable the model to produce text that aligns with and expands upon right-wing extremist prompts. With very little experimentation, short prompts produce compelling and consistent text that would believably appear in far-right extremist communities online,” the researchers wrote. “GPT-3’s ability to emulate the ideologically consistent, interactive, normalizing environment of online extremist communities poses the risk of amplifying extremist movements that seek to radicalize and recruit individuals. Extremists could easily produce synthetic text that they lightly alter and then employ automation to speed the spread of this heavily ideological and emotionally stirring content into online forums where such content would be difficult to distinguish from human-generated content.”

OpenAI says it’s experimenting with safeguards at the API level including “toxicity filters” to limit harmful language generation from GPT-3. For instance, it hopes to deploy filters that pick up antisemitic content while still letting through neutral content talking about Judaism.

Another solution might lie in a technique proposed by Salesforce researchers including former Salesforce chief scientist Richard Socher. In a recent paper, they describe GeDi (short for “generative discriminator”), a machine learning algorithm capable of “detoxifying” text generation by language models like GPT-3’s predecessor, GPT-2. During one experiment, the researchers trained GeDi as a toxicity classifier on an open source data set released by Jigsaw, Alphabet’s technology incubator. They claim that GeDi-guided generation resulted in significantly less toxic text than baseline models while achieving the highest linguistic acceptability.

But technical mitigation can only achieve so much. CTEC researchers recommend partnerships between industry, government, and civil society to effectively manage and set the standards for use and abuse of emerging technologies like GPT-3. “The originators and distributors of generative language models have unique motivations to serve potential clients and users. Online service providers and existing platforms will need to accommodate for the impact of the output from such language models being utilized with the use of their services,” the researchers wrote. “Citizens and the government officials who serve them may empower themselves with information about how and in what manner creation and distribution of synthetic text supports healthy norms and constructive online communities.”

It’s unclear the extent to which this will be possible ahead of the U.S. presidential election, but CTEC’s findings make apparent the urgency. GPT-3 and like models have destructive potential if not properly curtailed, and it will require stakeholders from across the political and ideological spectrum to figure out how they might be deployed both safely and responsibly.

For AI coverage, send news tips to Khari Johnson and Kyle Wiggers — and be sure to subscribe to the AI Weekly newsletter and bookmark our AI Channel.

Thanks for reading,

Kyle Wiggers

AI Staff Writer

Let’s block ads! (Why?)

Big Data – VentureBeat

Researchers find cutting-edge language models fall short in basic reasoning

September 9, 2020 Big Data

Even sophisticated language models such as OpenAI’s GPT-3 struggle with socially important topics like morality, history, and law. That’s the top-line finding from a new paper coauthored by Columbia, University of Chicago, and University of California, Berkeley researchers that proposes a 57-task test to measure models’ ability to reason. Models must possess problem-solving abilities and extensive knowledge about the world to perform well on the test. But in experiments, the coauthors found that the models they benchmarked — including GPT-3 — frequently didn’t know when they were wrong.

The goal of the novel test set is to bridge the gap between the knowledge that models see during training and existing measures of success in natural language processing. Like all machine learning models, language models learn patterns from vast data sets often sourced from Wikipedia, Reddit, ebooks, and other web sources. Some recently introduced benchmarks attempt to capture the linguistic skills of models, but so far, there’s little evidence to suggest a correlation between benchmark performance and a model’s grasp of commonsense reasoning.

The researchers claim their test is different in that it assesses models across subjects humans commonly learn, like mathematics, history, and ethics. To craft it, graduate and undergraduate students collected 15,908 questions from freely available sources online, including practice exams for undergraduate courses, quizzes for readers of Oxford University Press publications, and tests like the Graduate Record Examination, U.S. Medical Licensing Examination, and Examination for Professional Practice in Psychology. The tasks range in difficulty from an elementary level to an “advanced professional level,” a sampling the coauthors argue is sufficient for identifying a model’s blind spots.

Researchers find cutting edge language models fall short in basic reasoning

Above: Example questions from the researchers’ test set.

“We measure arbitrary real-world text understanding,” they wrote, noting that each subject contains at least 100 test examples. “Since models are pretrained on the internet, this enables us to test how well they can extract useful knowledge from massive corpora.”

In addition to GPT-3, the researchers benchmarked Google’s T5 and the Allen Institute for AI’s UnifiedQA question-answering model against their test set. The results show that meaningful progress has only become possible in recent months, with models containing up to 13 billion parameters achieving 25% accuracy and 175-billion-parameter models like GPT-3 reaching 43.9% accuracy. (Parameters are parts of the model learned from historical training data.) But that being the case, GPT-3 failed to excel at any single subject; its performance was on the test set was lopsided, with almost 70% accuracy for its best subject (U.S. foreign policy) but “near-random” performance for several other subjects (e.g., college chemistry).

“Overall, GPT-3 does poorly on highly procedural problems,” the researchers explained. “It is notably poor at modeling human (dis)approval, as evident by the low performance on the professional law and moral scenarios tasks, [and it] also has difficulty performing calculations, so much so that it exhibits poor performance on elementary mathematics and many other STEM subjects with ‘plug and chug’ problems … We speculate that is in part because GPT-3 acquires declarative knowledge more readily than procedural knowledge.”

The findings imply that current models have room for improvement, but it’s unclear whether existing techniques will suffice. As the researchers point out, previous research indicates that a 10 times increase in model size must be accompanied by an approximately 5 times increase in data, which might be logistically prohibitive.

“Aside from the tremendous expense in creating multi-trillion parameter language models, data may also become a bottleneck,” the researchers continued. “There is far less written about esoteric branches of knowledge than about everyday text.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Research shows natural language benchmarks don’t measure AI models’ general knowledge well

August 12, 2020 Big Data

Open-domain question-answering models — models theoretically capable of responding to novel questions with novel answers — often simply memorize answers found in the data on which they’re trained, depending on the data set. That’s the assertion of a team of researchers affiliated with Facebook and the University College London, who in a preprint paper present evidence that 60%-70% of answers given by models tested on open-domain benchmarks are embedded somewhere in the training sets.

Open-domain question-answering has received attention in the AI community for its practical applications, and more recently as a method to analyze language models’ grasp of factual knowledge. But a deep understanding of what kinds of questions models can answer remains elusive; unknowns about how questions and answers are distributed in benchmark corpora make it hard to contextualize the results.

In their study, the researchers sought to evaluate the test sets of popular open-domain question-answering data sets including WebQuestions, TriviaQA, and Open Natural Questions. They identified classes of question a model should be able to answer and annotated 1,000 question-answer pairs from each test set for repeated questions in their respective training sets. Then they computed the performance of several models on the benchmarks using open-book (which leverage retrieval from a large corpus of documents) and closed-book approaches (which focus on training large models with no external knowledge).

The three data sets in question aren’t much alike, which was the point — testing across all three guaranteed robustness. WebQuestions contains 3,778 training and 2,032 test question-answer pairs from a search engine, while TriviaQA has 78,785 training and 11,313 test question-answer pairs from free trivia websites. Meanwhile, Open Natural Questions comprises 79,168 training and 3,610 question-answer pairs from a combination of search engines and Wikipedia articles.

The team theorizes open-domain question-answering models should be able to (1) recall the answer to a question seen at training time, (2) answer novel questions at test time and choose an answer from the set of answers seen during training, and (3) answer novel questions that have answers not contained within the training data set. To determine whether the aforementioned benchmarks measure any of these behaviors, the coauthors split the test data in each corpus by whether the answers appeared somewhere in the training sets. Around 58%-71% of test answers were also somewhere in the training data, according to the researchers, demonstrating that the majority of the test data didn’t probe for answer generalization.

The team also probed the benchmarks for paraphrased questions in training data, using the set of 1,000 annotated questions. They say that 28%-34% of the questions were paraphrased, the majority being near-duplicates differing only by one or two words. “This result implies that 30% of the test set of these datasets only probe for how well models can simply memorize question-answer pairs seen at training,” the coauthors wrote.

The researchers selected several “open book” models — dense passage retrieval, retrieval-augmented generation, and fusion-in-decoder — and “closed book” models (Facebook’s BART and Google’s T5) to test, as well as nearest-neighbor models that store all available answers and classify new answers based on a similarity measure. Results on the benchmark corpora imply that all models memorized questions well, with an untrained nearest-neighbor model answering 20% of the test questions correctly. But they performed poorly on questions that couldn’t be memorized from training sets, with a mean absolute performance difference of 63% between repeated and non-repeated data. And when it came to generalization, one model that reliably memorized questions — T5 — struggled, achieving only a 22% match score.

“It is clear that performance on these data sets cannot be properly understood by overall question-answer accuracy,” the researchers wrote. “We suggest that in future, a greater emphasis be placed on more behavior-driven evaluation rather than pursuing single-number overall accuracy figures.”

Let’s block ads! (Why?)

Big Data – VentureBeat

AI Weekly: Can language models learn morality?

August 8, 2020 Big Data

The fervor around state-of-the-art AI language models like OpenAI’s GPT-3 hasn’t died down. If anything, it’s gaining steam. Melanie Mitchell, a professor of computer science at Portland State University, found evidence that GPT-3 can make primitive analogies. Raphaël Millière, a philosopher of mind and cognitive science at Columbia University’s Center for Science and Society, asked GPT-3 to compose a response to the philosophical essays written about it. Among other applications, the API providing access to the model has been used to create a recipe generator, an all-purpose Excel function, and a comedy sketch writer.

But even language models as powerful as GPT-3 have limitations that remain unaddressed. Morality aside, countless studies have documented their tendency to reinforce the gender, ethnic, and religious stereotypes explicit within the data sets on which they’re trained. Shortcomings like these could lead to headline-generating models with a negative slant against people of color, for example, or news-summarizing models with warped concepts of gender.

In an effort to highlight models’ ethical dilettantism, researchers at Microsoft; the University of California, Berkeley; Columbia University; and the University of Chicago coauthored a preprint paper that assesses language models’ knowledge of moral concepts. They claim the benchmark they devised — dubbed ETHICS — provides a stepping stone to AI that’s better aligned with human values.

Some scientists argue improvements in language processing won’t necessarily lead to ethical AI because intelligence is divorced from moral behavior. Others claim that while ethical AI will be an important problem in the future, it’s outside the scope of data science and machine learning capabilities today. In any case, few (if any) methods of measuring a natural language system’s grasp of human values currently exist, which is what motivated the study.

The coauthors note that fairness is a concept of justice that more broadly encompasses concepts like impartiality and desert. (In philosophy, “desert” is the condition of deserving something.) Having systems abide by safety constraints is similar to deontological ethics in which right and wrong are determined by a collection of rules. Imitating prosocial behavior and demonstrations is an aspect of virtue ethics, which locates moral behavior in the imitation of virtuous agents. And improving utility by learning human preferences can be viewed as part of utilitarianism, or the theory that advocates maximizing the aggregate well-being of all people. ETHICS attempts to tie these separate strands — justice, deontology, virtue ethics, utilitarianism, and commonsense moral judgments — together by confronting the challenges posed by open-world scenarios and covering applicable theories in normative ethics.

ETHICS requires models to learn how basic truths about the world connect with human values, like the fact that although everyone coughs, people don’t want to be coughed on because it might make them sick. It’s the researchers’ assertion this contextualized setup captures the type of nuance necessary for a more general understanding of ethical principles.

To perform well on the ETHICS data set’s over 130,000 scenarios, models must reason about morally relevant factors emphasized by each of several ethical systems. The scenarios regarding justice underline notions of impartiality. The deontological scenarios emphasize rules, obligations, and constraints. Character traits like benevolence and truthfulness are paramount in the virtue ethics examples. And while happiness or well-being are the sole factors for the utilitarian scenarios, both are involved in the commonsense moral intuition scenarios.

The researchers took steps to ensure that scenarios within ETHICS didn’t involve ambiguous moral dilemmas. (For instance, “I broke into a building” is treated as morally wrong in the ETHICS data set, even though there might be situations where it isn’t wrong, such as if you’re a firefighter trying to save someone from a burning building.) They had Amazon Mechanical Turk workers relabel each scenario and discard those scenarios with low agreement, collecting data from English speakers in the U.S., Canada, and Great Britain and focusing on uncontroversial topics.

Over the course of several experiments, the researchers tested leading language models, including Google’s BERT and ALBERT, Facebook’s RoBERTa, and GPT-3. They found that all four achieved low performance on most moral reasoning tasks — one BERT variant answered questions about justice with 11.9% to 15.2% accuracy — but bigger models trained on more data tended to do “significantly” better than smaller models. For instance, the largest RoBERTa model answered questions about the scenarios ethically 44.1% to 68% of the time, which was far better than chance (24.2%).

The researchers posit that aligning AI with human values appears difficult in part because those values contain preferences intertwined with subconscious desires. It’s also true that popular language models trained with large corpora demonstrate several forms of bias. Recently, Facebook AI head Jerome Pesenti found a rash of negative statements from GPT-3, including several that targeted Black people, Jewish people, and women. Emily Bender, a professor at the University of Washington’s NLP group, recently told VentureBeat that even carefully crafted language data sets can carry forms of bias.

The ETHICS work coauthors believe representations could imbue language models with a broader set of human preferences about the world. In tandem with techniques to mitigate the effects of prejudiced data, these representations could also bolster efforts within the AI research community to create more equitable, less potentially harmful applications of AI.

“Systems would do well to understand the ethical factors at play to make better decisions within the boundaries of the law,” the coauthors wrote. “Our work is just a first step that is necessary but not sufficient for creating ethical AI, as we must engage more stakeholders and successfully implement their values. Future work should also make sure these models are explainable, and should test model robustness to optimization pressure.”

Indeed, work to imbue models with morality is likely necessary on the path toward sophisticated AI assistants. In remarks at MIT’s Computing Community Consortium in March 2019, Eric Schmidt, former executive chairman of Google and Alphabet, described his vision of assistants of the future that might help children to learn language and math; help adults plan their day; and provide warmth and companionship to the elderly. If such assistants were to lack a moral compass of any kind, the impact could be harmful particularly on young children, who lack a nuanced understand of right and wrong.

Let’s block ads! (Why?)

Big Data – VentureBeat

Google’s AI tool lets users trigger mobile app actions with natural language instructions

July 11, 2020 Big Data

Take the latest VB Survey to share how your company is implementing AI today.

Google is investigating ways AI might be used to ground natural language instructions to smartphone app actions. In a study accepted to the 2020 Association for Computational Linguistics (ACL) conference, researchers at the company propose corpora to train models that would alleviate the need to maneuver through apps, which could be useful for people with visual impairments.

When coordinating efforts and accomplishing tasks involving sequences of actions — for example, following a recipe to bake a birthday cake — people provide each other with instructions. With this in mind, the researchers set out to establish a baseline for AI agents that can help with similar interactions. Given a set of instructions, these agents would ideally predict a sequence of app actions as well as the screens and interactive elements produced as the app transitions from one screen to another.

In their paper, the researchers describe a two-step solution comprising an action-phrase extraction step and a grounding step. Action-phrase extraction identifies the operation, object, and argument descriptions from multi-step instructions using a Transformer model. (An “area attention” module within the model allows it to attend to a group of adjacent words in the instruction as a whole for decoding a description.) Grounding matches the extracted operation and object descriptions with a UI object on the screen, again using a Transformer model but one that contextually represents UI objects and grounds object descriptions to them.

Google’s AI tool lets users trigger mobile app actions with natural language instructions

Above: The action-phrase extraction model takes a word sequence of a natural language instruction and outputs a sequence of spans (denoted in red boxes) that indicate the phrases describing the operation, the object, and the argument of each action in the task.

Image Credit: Google

The coauthors created three new data sets to train and evaluate their action-phrase extraction and grounding model:

The first contains 187 multi-step English instructions for operating Pixel phones along with their corresponding action-screen sequences.
The second contains English “how-to” instructions from the web and annotated phrases that describe each action.
The third contains 295,000 single-step commands to UI actions covering 178,000 UI objects across 25,000 mobile UI screens from a public Android UI corpus.

They report that a Transformer with area attention obtains 85.56% accuracy for predicting span sequences that completely match the ground truth. Meanwhile, the phrase extractor and grounding model together obtain 89.21% partial and 70.59% complete accuracy for matching ground-truth action sequences on the more challenging task of mapping language instructions to executable actions end-to-end.

The researchers assert that the data sets, models, and results — all of which which are available on open source on GitHub — provide an important first step on the challenging problem of grounding natural language instructions to mobile UI actions.

“This research, and language grounding in general, is an important step for translating multi-stage instructions into actions on a graphical user interface. Successful application of task automation to the UI domain has the potential to significantly improve accessibility, where language interfaces might help individuals who are visually impaired perform tasks with interfaces that are predicated on sight,” Google Research scientist Yang Li wrote in a blog post. “This also matters for situational impairment when one cannot access a device easily while encumbered by tasks at hand.”

Let’s block ads! (Why?)

Big Data – VentureBeat

OpenAI teaches its language model to classify and generate images

June 17, 2020 Big Data

In a paper accepted to the upcoming International Conference on Machine Learning (ICML) 2020 conference, researchers at OpenAI demonstrate that certain AI language models trained on pixel sequences can generate coherent images. They say it’s a small but significant step toward understanding and bridging the gap between computer vision and language understanding techniques.

Self-supervised learning, or learning without human-labeled data, is a longstanding challenge in machine learning. Recently, models like Google’s BERT, Facebook’s RoBERTa, and OpenAI’s GPT-3 have achieved leading performance on a range of language tasks, but this same emerging class hasn’t been successful when applied to image generation or classification.

Fortunately, Transformer-based models like GTP-3 are domain-agnostic, meaning they can be applied to sequences of any form. OpenAI exploited this to train a smaller version of its language model, GPT-2, on image data. The results indicate the model understands characteristics like object appearances and categories even without hand-coded knowledge; features from the model achieve state-of-the-art performance on a number of classification corpora and near state-of-the-art unsupervised accuracy.

OpenAI teaches its language model to classify and generate images

Above: Images generated by OpenAI’s iGPT models.

Image Credit: OpenAI

OpenAI trained three versions of image-generating GPT-2 models — iGPT-S (which contained 76 million parameters), iGPT-M (455 million parameters), and iGPT-L (1.4 billion parameters) — on the popular benchmark corpus ImageNet, and an even larger model dubbed iGPT-XL (6.8 billion parameters) on a mix of ImageNet and images from the web. They then reduced the images’ resolutions and created their own 9-bit color palette to represent pixels, yielding an input sequence length 3 times shorter than the standard RGB spectrum without sacrificing accuracy.

VB Transform 2020 Online – July 15-17. Join leading AI executives: Register for the free livestream.

According to OpenAI, the results show that image feature quality sharply increased during training before mildly decreasing. The researchers posit this might have occurred because Transformer-based models operate in two phases. In the first phase, the model gathers information from its surrounding context to build contextualized image features, and in the second phase, the contextualized features are used to predict the next pixels in images.

OpenAI also found that both increasing the scale of its models and training for more iterations resulted in better image quality. When the features were evaluated on the benchmarks CIFAR-10, CIFAR-100, and STL-10, they outperformed those from all supervised and unsupervised transfer algorithms.

Above: The iGPT’s benchmark performance.

Image Credit: OpenAI

However, OpenAI notes that their approach has limitations. Its iGPT models field only low-resolution images and exhibit biases that are a consequence of the data they’ve been trained on — for instance, perhaps developing associations between genders and roles (i.e., “male scientist”). Moreover, they take large amounts of time and compute to train — roughly 2,500 days for iGPT-L on an Nvidia V100 graphics card.

For this reason, the work primarily serves as a proof-of-concept demonstration, according to the researchers. “The significant resource cost to train these models and the greater accuracy of [existing] methods precludes these representations from practical real-world applications in the vision domain … [and] expect that developers will need to pay increasing attention to the data that they feed into their systems and to better understand how it relates to biases in trained models,” they wrote. “[However, our] results suggest that due to its simplicity and generality, a sequence transformer given sufficient compute might ultimately be an effective way to learn excellent features in many domains.”

OpenAI has long asserted that powerful computers in conjunction with reinforcement learning and other techniques can achieve paradigm-shifting AI advances. As MIT Technology Review reported earlier this year, a team within OpenAI called Foresight runs experiments to test how far they can push AI capabilities by training algorithms with increasingly large amounts of data and compute. According to that same report, OpenAI is developing a system trained on images, text, and other data using massive computational resources that the company’s leadership believes is the most promising path toward artificial general intelligence (AGI), or AI that can learn any task a human can.

Let’s block ads! (Why?)

Big Data – VentureBeat

Language may help AI navigate new environments

April 19, 2020 Big Data

In a new study published this week on the preprint server Arxiv.org, scientists at the University of Toronto and the Vector Institute, an independent nonprofit dedicated to advancing AI, propose BabyAI++, a platform to study whether descriptive texts help AI to generalize across dynamic environments. Both it and several baseline models will soon be available on GitHub.

One of the most powerful techniques in machine learning — reinforcement learning, which entails spurring software agents toward goals via rewards — is also one of the most flawed. It’s sample inefficient, meaning it requires a large number of compute cycles to complete, and without additional data to cover variations, it adapts poorly to environments that differ from the training environment.

It’s theorized that prior knowledge of tasks through structured language could be combined with reinforcement learning to mitigate its shortcomings, and BabyAI++ was designed to put this theory to the test. To this end, the platform builds upon an existing reinforcement learning framework — BabyAI — to generate various dynamic, color tile-based environments along with texts that describe their layouts in detail.

Language may help AI navigate new environments

Above: Environments in BabyAI++.

BabyAI++’s levels consist of objects that can be picked up and dropped; doors that can be unlocked and opened; and various tasks that the agents must be undertake. Like the environments themselves, the tasks are randomly generated, and they’re communicated to the agent through “Baby-Language,” a compositional language that uses a subset of English vocabulary.

The abovementioned texts reveal which types of tiles are in use and what color is matched to each tile. Since the pairing between the color and tile type is randomized, the agent must understand the description for it to properly navigate the map.

Within BabyAI++, every level is partitioned into two configurations: training and testing. In the training configuration, the agent is exposed to all tile and colors types in the level, but some combinations of color-type pairs are held out. In the testing configuration, all color-type pairs are enabled, forcing the agent to use language grounding to associate the type of the tile to the color.

The paper describes several experiments that were conducted using the baseline models, one of which — attention-fusion — uses what’s called an attention mechanism to assign relevant text embeddings (mathematical representations) to locations on a scene embedding feature map (a function that maps embeddings to a feature space, or the dimensions where the variables the AI processes reside). For the most difficult level, this attention-fusion model had a 16.2% higher testing success rate (around 60% after 5 steps, or actions) than the second-best model on the most challenging level, and it completed the level using fewer frames of images (around 65 compared with 75).

Above: The attention model’s architecture.

The coauthors assert that this shows descriptive texts are useful for agents to generalize environments with variable dynamics by learning language-grounding. “We believe the proposed BabyAI++ platform, with its public code and baseline implementations, will further spur research development in this area,” they wrote in the paper.

Let’s block ads! (Why?)

Big Data – VentureBeat

MIT CSAIL’s TextFooler generates adversarial text to strengthen natural language models

February 9, 2020 Big Data

AI and machine learning algorithms are vulnerable to adversarial samples that have alterations from the originals. That’s especially problematic as natural language models become capable of generating humanlike text, because of their attractiveness to malicious actors who would use them to produce misleading media. In pursuit of a technique that illustrates the extent to which adversarial text can affect model prediction, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the University of Hong Kong, and Singapore’s Agency for Science, Technology, and Research developed TextFooler, a baseline framework for synthesizing adversarial text examples. They claim in a paper that it was able to successfully attack three leading target models, including Google’s BERT.

“If those tools are vulnerable to purposeful adversarial attacking, then the consequences may be disastrous,” said Di Jin, MIT Ph.D. student and lead author on the paper, who noted that the adversarial examples produced by TextFooler could improve the robustness of AI models trained on them. “These tools need to have effective defense approaches to protect themselves, and in order to make such a safe defense system, we need to first examine the adversarial methods.”

The researchers assert that besides the ability to fool AI models, the outputs of a natural language “attacking” system like TextFooler should meet certain criteria: human prediction consistency, such that human predictions remain unchanged; semantic similarity, such that crafted examples bear the same meaning as the source; and language fluency, such that generated examples look natural and grammatical. TextFooler meets all three even when no model architecture or parameters (values that influence model performance) are available — i.e., black-box scenarios.

It achieves this by identifying the most important words for the target models and replacing them with semantically similar and grammatically correct words until the prediction is altered. TextFooler is applied to two different tasks — text classification and entailment (the relationship between text fragments in a sentence) — with the goal of changing the classification or invalidating the entailment judgment of the original models. For instance, given the input “The characters, cast in impossibly contrived situations, are totally estranged from reality,” TextFooler might output “The characters, cast in impossibly engineered circumstances, are fully estranged from reality.”

To evaluate TextFooler, the researchers applied it to text classification data sets with various properties, including news topic classification, fake news detection, and sentence- and document-level sentiment analysis, where the average text length ranged from tens of words to hundreds of words. For each data set, they trained the aforementioned state-of-the-art models on a training set before generating adversarial examples semantically similar to the test set to attack those models.

The team reports that on the adversarial examples, they managed to reduce the accuracy of almost all target models in all tasks to below 10% with fewer than 20% of the original words perturbed. Even for BERT, which attained relatively robust performance compared with the other models tested, TextFooler reduced its prediction accuracy by about 5 to 7 times on a classification task and about 9 to 22 times on an entailment task (where the goal was to judge whether a sentence could be derived from entailment, contradiction, or a neutral relationship).

“The system can be used or extended to attack any classification-based NLP models to test their robustness,” said Jin. “On the other hand, the generated adversaries can be used to improve the robustness and generalization of deep learning models via adversarial training, which is a critical direction of this work.”

Let’s block ads! (Why?)

Big Data – VentureBeat

Storage 101: The Language of Storage

February 2, 2020 BI News and Info

The series so far:

Storage drives come in many shapes and sizes, and it can be difficult to distinguish one from the other beyond their appearances because vendor-supplied information is sometimes confusing and obscure. Although the material has gotten better over the years, it can still be unclear. Yet understanding this information is essential to knowing how well a drive will perform, how much data it will hold, its expected lifespan, and other important features.

In the first article in this series, I introduced you to a number of storage-related concepts, all of which can play an important role in determining what each drive offers and how they differ. For example, a solid-state drive (SSD) that uses the Peripheral Component Interconnect Express (PCIe) interface will typically perform better than one that uses the Serial Advanced Technology Attachment (SATA) interface, and a SATA-based SSD will likely perform better than a SATA-based hard-disk-drive (HDD).

But the interface and drive type are only part of the equation when it comes to choosing storage media. You must also take into account latencies, input/output operations per second (IOPS), throughput, effective and usable capacities, data transfer size or I/O size, endurance, and other factors. Unfortunately, it’s no small matter trying to make sense of all these variables, especially with the inconsistencies among storage vendors in how they present information about their products.

In this article, I dig into concepts commonly used to describe storage media to help make sense of the information you’ll encounter when evaluating HDDs or SSDs for your organization. Many of the concepts are specific to performance, but I also discuss issues related to capacity and lifespan, particularly in how they differ between HDDs and SSDs.

Making Sense of Performance Metrics

When architecting storage solutions to meet enterprise requirements such as performance, you should identify the workloads that the devices must support. To this end, you must understand data access patterns such as read operations versus writes, random operations versus sequential, and block transfer size.

In this regard, storage operations can be divided into four types: random reads, random writes, sequential reads, and sequential writes. In some cases, these operations can be further divided by the block transfer size (small versus large), which depends on the application. Many workloads use a mix of two or more of these types, although they might favor one over the others. Common data access patterns include:

Random read/write, small block: A wide variety of applications such as transactional business applications and associated databases.
Sequential write, large block: Loading media, loading data warehouse.
Sequential read, large block: Reading media, data warehouse aggregations and reporting.
Sequential write, small block: Database log writes.
Mixed: Any combination of the above. Note that, when multiple sequential workloads are active concurrently, the workload becomes randomized.

When shopping for storage, you’ll encounter an assortment of metrics that describe how well a drive is expected to perform. Understanding these metrics is essential to ensuring that you’re using the right drives to support your specific workloads.

Latency refers to a drive’s response time, that is, how long it takes for an I/O operation to complete. From an application’s perspective, latency is the time between issuing a request and receiving a response. From a user perspective, latency is the only metric that matters.

Vendors list latency in milliseconds (ms) or microseconds (µs). The lower the number, the shorter the wait times. However, the rate can quickly increase as individual I/O requests start piling up, the I/O sizes increase (which typically range between 4 KB to 512 KB), or the nature of the workload changes, such as changing from read-only to read/write or from sequential to random. For example, a drive’s latency might be listed as 20ms, but if the drive is supporting concurrent read operations, I/O requests could end up in a long queue, causing a dramatic increase in latency.

Latency is nearly always a useful metric when shopping for drives and should be considered in conjunction with IOPS. For example, a storage solution that provides 185 IOPS with an average latency of 5ms might deliver better application performance than a drive that offers 440 IOPS but with 30ms latency. It all depends on the workloads that the drives will need to support.

Another common metric is IOPS, which indicates the maximum number of I/O operations per second that the drive is expected to support. An I/O operation is the transfer of data to or from the drive. The higher the number of supported IOPS, the better the performance—at least according to conventional wisdom. In truth, IOPS tells only part of the story and should be considered along with other important factors, such as latency and I/O size. IOPs is most relevant for random data access patterns (and far less important for sequential workloads).

Another important metric is throughput, which measures the amount of data that can be written to or read from a storage drive within a given timeframe. Some resources may refer instead to data transfer rate or simply transfer rate, sometimes according to drive type. For example, you might see transfer rate used more often with HDDs and throughput associated with SSDs. Like other performance metrics, throughput is dictated by the nature of the workload. Throughput is most relevant for sequential data access patterns (and far less relevant for random workloads).

The distinction between sequential and random is particularly important for HDDs because of how data is stored on the platters, although it can also play a role in SSD performance. In fact, there are other differences between the two drive types that you need to understand when evaluating storage. Discussions of device capacity and endurance also follow.

What Sets HDDs Apart

Data is written to an HDD in blocks that are stored sequentially or scattered randomly across a platter. Enterprise drives contain multiple platters with coordinated actuator arms and read/write heads that move across the platters (a topic I’ll be covering more in-depth later in the series).

Whenever an application tries to access the data, the platter’s actuator arm must move the head to the correct track and the platter must be rotated to the correct sector. The time required to do so is referred to as the seek time. The time it takes for the platter to rotate to the correct sector is referred to as the rotational latency.

The duration of an I/O operation depends on the location of the head and platter prior to the request. When the data blocks are saved sequentially on the disk, an application can read and write data in relatively short order, reducing seek times and rotational latencies to practically nothing. If the blocks are strewn randomly across the platters, every operation requires the actuator to move the head to a different area on the platter, resulting in rotational latency and seek time, and therefore slower performance.

Because of these differences, it is essential to evaluate HDDs in terms of the workloads you plan to support, taking into account such factors as file size, concurrency, and data access patterns (random vs. sequential, read vs. write, big block vs. small block, and mixed). By taking your workloads into account, you’ll have a better sense of how to select the right drives for the workloads demanded by your organization’s applications.

For example, Dell offers a 12-TB SATA drive that supports up to 118 IOPS for random reads and 148 IOPS for random operations that include 70% reads and 30% writes at a given latency. Whereas the same drive offers a throughput of 228 MB/s for sequential read and write operations. From these metrics, you can start getting a sense of whether the drive can meet the needs of your anticipated workloads.

Suppose you’re looking for a storage solution to support a set of read-intensive web applications whose storage access pattern is primarily random reads, as opposed to sequential reads. You would want to compare the Dell drive against other drives to determine which one might offer the best IOPS, with less emphasis placed on the other types of access patterns.

One characteristic driving HDD performance is revolutions per minute (RPM), that is, the number of times the drive’s platters rotates within a minute. The higher the number of RPMs, the faster the data can be accessed, leading to lower latency rates and higher performance. Enterprise-class HDDs typically support 10,000 or 15,000 RPMs, often written as simply 10K or 15K RPMs.

You must also take into account a drive’s available capacity, keeping in mind that you never load an HDD anything close to its physical capacity.

A drive’s anticipated lifespan is indicated by the mean time between failures (MTBF) rating, the number of operating hours expected before failure. For example, Dell offers several 14-TB and 16-TB drives with MTBF ratings of 2,500,000 hours, which comes to over 285 years. Such ratings are common, yet in reality drives fail far more frequently than high MTBF suggests. MTBF is only a small part of the reliability equation. Enterprise solutions demand the identification and elimination of single points of failure and redundancies across components, starting at the drive level.

By looking at the various metrics, you have a foundation to begin comparing drives. Historically, marketing considerations—trying to present their drives in the best light—resulted in presenting performance specs in a way that made comparisons across vendors challenging. Today’s specifications are generally more consistent, at least enough to make reasonable apples-to-apples comparisons. Some vendors also provide insights beyond the basics, for example, providing performance metrics featuring a mix of workloads, such as 70% reads and 30% writes at a given latency.

What Sets SSDs Apart

Consumer and enterprise SSDs are based on NAND flash technologies, a type of nonvolatile memory in which data is stored by programming integrated circuit chips, rather than manipulating magnetic properties, as with an HDD. Also unlike the HDD, the SSD has no moving parts, which makes reading and writing data much faster operations.

If you’re relatively new to enterprise SSDs, the terminology that surrounds these drives can be confusing. Yet the fundamental performance considerations are exactly the same: latency, IOPs, throughput, capacity, and endurance .

As with an HDD, capacity in an SSD refers to the amount of data it can hold. With SSDs, however, vendors often toss around multiple terms related to capacity and do so inconsistently, so it’s not always clear what each one means. For example, some vendors list a drive’s total capacity as raw capacity or just capacity, both of which refer to the same thing—the maximum amount of raw data that the drive can hold.

Not all of the drive’s raw capacity is available for storing data. The drive must be able to accommodate the system overhead required to support various internal SSD operations. For this reason, the amount of available capacity is always less than the amount of raw capacity. Vendors often refer to available capacity as usable capacity.

Conceptually, raw capacity and usable capacity are similar between HDDs and SSDs. For example, when calculating capacities, you should take into account that some of that space must be available for system data, as well as for recovery configurations such as RAID.

Another characteristic that sets the SSD apart is the way in which bits are stored in the flash memory cells. Today’s SSDs can store up to four bits per cell, with talk of five bits in the wings. Vendors often reference the bit-level in the drive’s specifications. For example, a drive that supports three bits per cell is referred to as triple-level cell (TLC) flash, and a drive that supports four bits per cell is referred to as quad-level cell (QLC) flash.

In addition to squeezing more bits into a cell, vendors are also trying to get more cells onto a NAND chip. The more bits the chip can support, the greater the data density.

As with HDDs, SSD endurance refers to the drive’s longevity or lifespan; however, it’s measured differently. SSD drive endurance is based on the number of program/erase cycles (P/E cycles) it will support. NAND cells can tolerate only a limited number of P/E cycles. The higher that number, the greater the endurance. A drive’s endurance is measured by its write operations because read operations have minimal impact on an SSD. Whereas the HDD endurance metric is MTBF, vendors report an SSD’s endurance by providing the number of drive writes per day (DWPD), terabytes written (TBW), or both. The DWPD metric refers to the number of times you can completely overwrite the drive’s capacity each day during its warranted lifespan. The TBW metric indicates the total number of write operations that the drive will support over its lifetime. The DWPD and TBW are quite useful for comparing drives.

As vendors squeeze more bits into each cell and more cells into each chip, SSD vendors incorporate sophisticated technologies to mitigate the challenges of maintaining data integrity concomitant with higher data densities. For example, all SSDs employ wear leveling, over-provisioning, garbage collection, and error correction code (ECC) to extend the drive’s life, all of which I’ll be covering in more detail later in the series.

Moving toward a better understanding of storage

You should use the vendors’ published information only as a starting point for evaluating drives and identifying candidates. Consider additional research such as community reviews, benchmarks, or other resources to give you a realistic understanding of a drive’s capabilities. Your goal is to get as complete a picture as possible before investing in and testing solutions.

In addition, when comparing storage solutions, be sure to take into account the effective capacity, which is the amount of storage available to a drive after applying data-reduction technologies such as deduplication and compression. Such data-reduction strategies make it possible for the drive to store much more data. For example, IBM offers a flash-based storage system that provides 36.9 TB of raw capacity but supports up to 110 TB of effective capacity

Clearly, you need to understand a wide range of concepts to determine what types of storage will best support your workloads. Not only must you choose between HDDs and SSDs, but you must also select from within these categories, while taking into account such factors as latency, IOPs, throughput, densities, and numerous others considerations.

In the articles to follow, I’ll be digging into HDDs and SSDs more deeply so you can better understand their architectures and how they differ in terms of capacity, performance, and endurance. With this foundation, you’ll be better equipped to determine what storage solutions you might need for your organization, based on your applications and workloads.

Let’s block ads! (Why?)

SQL – Simple Talk

Tag Archives: language

Allen Institute researchers find pervasive toxicity in popular language models

AI Weekly: Cutting-edge language models can produce convincing misinformation if we don’t stop them

Researchers find cutting-edge language models fall short in basic reasoning

Research shows natural language benchmarks don’t measure AI models’ general knowledge well

AI Weekly: Can language models learn morality?

Google’s AI tool lets users trigger mobile app actions with natural language instructions

OpenAI teaches its language model to classify and generate images

Language may help AI navigate new environments

MIT CSAIL’s TextFooler generates adversarial text to strengthen natural language models

Storage 101: The Language of Storage

Making Sense of Performance Metrics

What Sets HDDs Apart

What Sets SSDs Apart

Moving toward a better understanding of storage

Recent Posts

Categories

Archives