Ethan Yang (‘27) is a Mechanical Engineering and Applied Mechanics major currently pursuing a submatriculated Master of Science and Engineering (MSE) in Robotics. He conducted research on how humans interpret and perceive AI-generated audio with mentorship from Dr. Jianjing Kuang (Department of Linguistics). This research was supported by the Penn Undergraduate Research Mentoring Program.
As someone who trained for over a decade as a classical violinist, I have always seen phrasing – how sound rises, falls, and pauses – as its own kind of language. So, when I came across a PURM project investigating how machines speak and how humans interpret that speech, I saw it not as a detour from music, but as a natural extension of it.
Last summer, I collected human speech data through software, analyzed synthetic voices from models like OpenAI and Amazon, and began exploratory work for a perception study. My role spanned recruiting and guiding participants through recording sessions, troubleshooting equipment, and creating visualizations of pitch, duration, and intensity patterns in human and AI-generated audio. Whether I was in the lab adjusting the mics or at my laptop examining survey feedback from early perception trials, I was drawn to the interplay between how speech is produced and understood.
These recording sessions became a meaningful way to connect with the Penn community. I met participants from various disciplines, guided them through the recording process, and often found conversations before and after sessions to be as valuable as the recordings themselves. My mentor helped me see the broader research arc and encouraged me to share my questions and ideas. At times, I suggested different approaches, leading to constructive discussions that enhanced our methods. Engaging in this kind of open exchange with my mentor and peers was very rewarding, as it showed me how much stronger research becomes when every voice is heard.
This experience strengthened my technical and interpersonal skills. Technically, I learned to generate and evaluate synthetic speech from multiple AI and text-to-speech systems, run early-stage perception experiments, and integrate acoustic and perceptual data. Interpersonally, I honed my ability to engage participants, communicate clearly, and collaborate effectively. I also became more adaptable and more confident in advocating for different approaches than the ones initially proposed to help solve unexpected challenges.
I plan to carry these skills into ongoing work in the lab, applying them to future studies bridging AI, language, and human experience. The adaptability, analytical mindset, and collaborative habits I developed will help me contribute more deeply to our current project and prepare me to explore larger questions about how humans and machines understand one another.
Interested in reading more first-hand accounts about undergraduate research? Check out the other experiences featured on our Student News Page and Social Media!
Related Articles
More Than Just Robotics Research
Jeffrey Oduman (‘28), a Computer Science major, conducted robotics research with mentorship from Dr. Cynthia Sung (Department of Mechanical Engineering and Applied Mechanics). This research was supported by the Penn Undergraduate Research Mentoring Program.
Connections and Community
Hannah Shumsky (‘26), a Neuroscience major, conducted research on the effect of circadian rhythms on lung repair and regeneration with mentorship from Dr. Shaon Sengupta (Division of Neonatology, Children’s Hospital of Philadelphia). This research was supported by the College Alumni Society Undergraduate Research Grant.