Natural language generation (NLG) has seen remarkable success benefiting from the development of deep learning techniques. However, the prevalent neural models function like a black box; they do not explicitly model the underlying factors behind the observed text, such as syntax or sentiment. As a result, such models are not conducive to interpretation and do not allow control over the generation process. On the other hand, the expensive economical and environmental cost to train and deploy huge models makes it questionable whether scaling up model parameters is the only direction to enhance NLG models. In this talk, I will cover my research efforts towards addressing these challenges as: (1) developing interpretable and controllable generative text models, and (2) strengthening parametric NLG models through the lens of external, non-parametric memories. At the end of this talk I will briefly describe my visions on future work.

Junxian He is a fifth-year Ph.D. student in the school of computer science at Carnegie Mellon University, co-advised by Prof. Graham Neubig and Prof. Taylor Berg-Kirkpatrick. His research focuses on natural language processing, in particular on generative modeling and text generation. Before joining CMU, he obtained the bachelor degree from Shanghai Jiao Tong University. He is one of the ten recipients worldwide of 2020 Baidu PhD fellowship.