National Science Foundation Award IIS-1750383, June 2018–May 2023. “CAREER: Integrating perceptual models of auditory importance into deep learning-based noise-robust speech recognition.” PI: Michael Mandel. $497,162. Grant Description: Hearing is central to human interaction, but the hearing process is not easily observed. The objective of this project is to train models to identify portions of speech utterances that are important to their being correctly identified by human listeners, and to use predictions from these models to make automatic speech recognition (ASR) systems more noise robust by focusing on those regions. The ability to identify important regions of an utterance could significantly advance our understanding of healthy and impaired hearing. Improvements in automatic speech recognition would have broader impacts on the 260 million Americans who use smartphones and the $100 billion ASR industry. The educational portion of this project utilizes examples from speech, language, audio, and music processing to attract and retain students in Brooklyn College’s introductory programming course serving a diverse student body along with similar efforts at affiliated high school programs.
