• Home
  • About Us
  • Contact Us
  • Privacy Policy
  • Special Offers
Business Intelligence Info
  • Business Intelligence
    • BI News and Info
    • Big Data
    • Mobile and Cloud
    • Self-Service BI
  • CRM
    • CRM News and Info
    • InfusionSoft
    • Microsoft Dynamics CRM
    • NetSuite
    • OnContact
    • Salesforce
    • Workbooks
  • Data Mining
    • Pentaho
    • Sisense
    • Tableau
    • TIBCO Spotfire
  • Data Warehousing
    • DWH News and Info
    • IBM DB2
    • Microsoft SQL Server
    • Oracle
    • Teradata
  • Predictive Analytics
    • FICO
    • KNIME
    • Mathematica
    • Matlab
    • Minitab
    • RapidMiner
    • Revolution
    • SAP
    • SAS/SPSS
  • Humor

AI generates melodies from lyrics

September 1, 2019   Big Data
 AI generates melodies from lyrics

Generating sequences of musical notes from lyrics might sound like the stuff of science fiction, but thanks to AI, it might someday become as commonplace as internet radio. In a paper published on the preprint server Arxiv.org (“Conditional LSTM-GAN for Melody Generation from Lyrics“), researchers from the National Institute of Informatics in Tokyo describe a machine learning system that’s able to generate “lyrics-conditioned” melodies from learned relationships between syllables and notes.

“Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables to learn and discover latent relationship between interesting lyrics and accompanying melody,” wrote the paper’s coauthors. “With the development of available lyrics and melody dataset and [AI], musical knowledge mining between lyrics and melody has gradually become possible.”

As the researchers explain, notes have two musical attributes: pitch and duration. Pitches are perceptual properties of sounds that organize music by highness or lowness on a frequency-related scale, while duration represents the length of time that a pitch or tone is sounded. Syllables align with melodies in the MIDI files of music tracks; the columns within said files represent one syllable with its corresponding note, note duration, and rest.

The researchers’ AI system made use of the alignment data with a long-short-term memory (LSTM) network, a type of recurrent neural network capable of learning long-term dependencies, with a generative adversarial network (GAN), a two-part neural network consisting of generators that produce samples and discriminators that attempt to distinguish between the generated samples and real-world samples. The LSTM was trained to learn a joint embedding (mathematical representation) at the syllable and word levels to capture the synaptic structures of lyrics, while the GAN learned over time to predict melody when given lyrics while accounting for the relationship between lyrics and melody.

To train it, the team compiled a data set consisting of 12,197 MIDI files, each paired with lyrics and melody alignment — 7,998 files from the open source LMD-full MIDI Dataset and 4,199 from a Reddit MIDI dataset — which they cut down to 20-note sequences. They took 20,934 unique syllables and 20,268 unique words from the LMD-full MIDI, and extracted the beats-per-minute (BPM) value for each MIDI file, after which they calculated note durations and rest durations.

Here’s one generated sample:


https://venturebeat.com/wp-content/uploads/2019/08/gen1.wav

And here’s another:

https://venturebeat.com/wp-content/uploads/2019/08/gen3.wav

After splitting the corpus into training, validation, and testing sets and feeding them into the model, the coauthors conducted a series of tests to determine how well it predicted melodies sequentially aligned with the lyrics, MIDI numbers, note duration, and rest duration. They report that their AI system not only outperformed a baseline model “in every respect,” but that it approximated well to the distribution of human-composed music. In a subjective evaluation during which volunteers were asked to rate the quality of 12 20-second melodies generated using the baseline method, the AI model, and ground truth, scores given to melodies generated by the proposed model were closer to those composed by humans than the baseline.

The researchers leave to future work synthesizing melodies with sketches of uncompleted lyrics and predicting lyrics when given melodies as a condition.

“Melody generation from lyrics in music and AI is still unexplored well [sic],” wrote the researchers. “Making use of deep learning techniques for melody generation is a very interesting research area, with the aim of understanding music creative activities of human.”

AI might soon become an invaluable tool in musicians’ compositional arsenals, if recent developments are any indication. In July, Montreal-based startup Landr raised $ 26 million for a product that analyzes musical styles to create bespoke sets of audio processors, while OpenAI and Google earlier this year debuted online creation tools that tap music-generating algorithms. More recently, researchers at Sony investigated a machine learning model for conditional kick-drum track generation.

Let’s block ads! (Why?)

Big Data – VentureBeat

from, generates, Lyrics, melodies
  • Recent Posts

    • Someone’s having surgery
    • C’mon hooman
    • Build and Release Pipelines for Azure Resources (Logic Apps and Azure Functions)
    • Database version control: Getting started with Flyway
    • Support CRM with New Dynamics 365 Field Service Mobile App
  • Categories

  • Archives

    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
© 2021 Business Intelligence Info
Power BI Training | G Com Solutions Limited