• Home
  • About Us
  • Contact Us
  • Privacy Policy
  • Special Offers
Business Intelligence Info
  • Business Intelligence
    • BI News and Info
    • Big Data
    • Mobile and Cloud
    • Self-Service BI
  • CRM
    • CRM News and Info
    • InfusionSoft
    • Microsoft Dynamics CRM
    • NetSuite
    • OnContact
    • Salesforce
    • Workbooks
  • Data Mining
    • Pentaho
    • Sisense
    • Tableau
    • TIBCO Spotfire
  • Data Warehousing
    • DWH News and Info
    • IBM DB2
    • Microsoft SQL Server
    • Oracle
    • Teradata
  • Predictive Analytics
    • FICO
    • KNIME
    • Mathematica
    • Matlab
    • Minitab
    • RapidMiner
    • Revolution
    • SAP
    • SAS/SPSS
  • Humor

Google open-sources BERT, a state-of-the-art training technique for natural language processing

November 2, 2018   Big Data
 Google open sources BERT, a state of the art training technique for natural language processing

Natural language processing (NLP) — the subcategory of artificial intelligence (AI) that spans language translation, sentiment analysis, semantic search, and dozens of other linguistic tasks — is easier said than done. Procuring diverse datasets large enough to train text-parsing AI systems is an ongoing challenge for researchers; modern deep learning models, which mimic the behavior of neurons in the human brain, improve when trained on millions, or even billions, of annotated examples.

One popular solution is pretraining, which refines general-purpose language models trained on unlabeled text to perform specific tasks. Google this week open-sourced its cutting-edge take on the technique — Bidirectional Encoder Representations from Transformers, or BERT — which it claims enables developers to train a “state-of-the-art” NLP model in 30 minutes on a single Cloud TPU (tensor processing unit, Google’s cloud-hosted accelerator hardware) or a few hours on a single graphics processing unit.

The release is available on Github, and includes pretrained language representation models (in English) and source code built on top of the Mountain View company’s TensorFlow machine learning framework. Additionally, there’s a corresponding notebook on Colab, Google’s free cloud service for AI developers,

As Jacob Devlin and Ming-Wei Chang, research scientists at Google AI, explained, BERT is unique in that it’s both bidirectional, allowing it to access context from both past and future directions, and unsupervised, meaning it can ingest data that’s neither classified nor labeled. That’s as opposed to conventional NLP models such as word2vec and GloVe, which generate a single, context-free word embedding (a mathematical representation of a word) for each word in their vocabularies.

BERT learns to model relationships between sentences by pretraining on a task that can be generated from any corpus, Devlin and Chang wrote. It builds on Google’s Transformer, an open source neural network architecture based on a self-attention mechanism that’s optimized for NLP. (In a paper published last year, Google showed that Transformer outperformed conventional models on English to German and English to French translation benchmarks while requiring less computation to train.)

When tested on the Stanford Question Answering Dataset (SQuAD), a reading comprehension dataset comprising questions posed on a set of Wikipedia articles, BERT achieved 93.2 percent accuracy, besting the previous state-of-the-art and human-level scores of 91.6 percent and 91.2 percent, respectively. And on the General Language Understanding Evaluation (GLUE) benchmark, a collection of resources for training and evaluating NLP systems, it hit 80.4 percent accuracy.

The release of BERT follows on the heels of the debut of Google’s AdaNet, an open source tool for combining machine learning algorithms to achieve better predictive insights, and ActiveQA, a research project that investigates the use of reinforcement learning to train AI agents for question answering.

Let’s block ads! (Why?)

Big Data – VentureBeat

a Training, BERT, Google, language, Natural, opensources, processing, stateoftheart, technique
  • Recent Posts

    • Derivative of a norm
    • TODAY’S OPEN THREAD
    • IBM releases Qiskit modules that use quantum computers to improve machine learning
    • Transitioning to Hybrid Commerce
    • Bad Excuses
  • Categories

  • Archives

    • April 2021
    • March 2021
    • February 2021
    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
© 2021 Business Intelligence Info
Power BI Training | G Com Solutions Limited