Conditional LSTM-GAN for Melody Generation from Lyrics
Speaker
Yi Yu, National Institute of Informatics
Time
2019-11-14 14:00:00 ~ 2019-11-14 15:30:00
Location
Room 1319, Software Expert Building
Host
Quanshi Zhang, Associate Professor, John Hopcroft Center for Computer Science
Abstract
Generating melody from lyrics to compose a song has been a very interesting research topic in the area of artificial intelligence and music, which tries to predict generative music relationship between lyrics and melody. Unfortunately, the limited availability of paired lyrics-melody corpus with alignment information has hindered the research progress. To address this problem, a large corpus consisting of 12,197 MIDI songs each with paired English lyrics and melody alignment is created by our group in National Institute of Informatics (NII), Tokyo. In the presentation, I will introduce a novel deep generative model, conditional Long Short-Term Memory - Generative Adversarial Network (LSTM-GAN) for melody generation from lyrics, which is developed based on this large corpus. Moreover, some interesting experiment and demonstration will confirm that plausible and tuneful melody can be inferred from lyrics.
Bio
Dr. Yi Yu is an assistant professor with National Institute of Informatics (NII). Before joining NII, she was a senior research fellow at School of Computing, National University of Singapore. Her research interests include generative modeling and embedding learning, and its application in data mining, vision, music, and multimedia. She has published more than 60 scientific publications, and has supervised more than 30 students including Ph.D students, Master students, and project researchers. She and her team received best poster award from IEEE ISM 2018 and best paper runner up in APWeb-WAIM 2017, were recognized as finalist of the World’s FIRST 10K best paper award in ICME17, won the 2nd prize in Yahoo Flickr grand challenge 2015, were in the top winners (out of 29 teams) awarded by ACM SIGSPATIAL GIS Cup 2013, and received a best paper award from IEEE ISM 2012.
Personal webpage: http://research.nii.ac.jp/~yiyu/.
Personal webpage: http://research.nii.ac.jp/~yiyu/.