Congratulations to Professor Nanyun (Violet) Peng on receiving the NSF CAREER Award. Language models (LMs) are fundamental to most natural language processing (NLP) applications today. However, prevalent auto-regressive models, which predict the next word based on left-side context, pose limitations in generation efficiency, control for human-machine collaboration, and human-like sentence composition. To address these challenges, Professor Peng’s research explores insertion-based models, which iteratively insert words into incomplete contexts, offering enhanced flexibility and control. This paradigm better mirrors human writing behaviors, benefiting computational linguistics and cognitive science for language structure studies.
The Insertion-Based Natural Language Generation project aims to advance understanding of insertion-based LMs by exploring optimal generation orders guided by linguistic theories and introducing a novel architecture incorporating deletion operations to rectify errors. Additionally, the project is investigating scaling up pre-training of these models. Success could lead to a family of large LMs surpassing auto-regressive counterparts in flexibility, controllability, and efficiency, benefiting various NLP tasks.
Currently, as an assistant professor at the Computer Science Department, Peng focuses on building robust NLP tools to lower communication barriers and enable AI companionship. Her research spans creative language generation, low-resource information extraction, and zero-shot cross-lingual transfer.
This award underscores NSF’s commitment to supporting impactful research, evaluated based on intellectual merit and broader impacts.