Skip to main content

Undergraduate Program

Summer REU

National Science Foundation Research Experience for Undergraduates (NSF REU)

Computational Methods for Understanding Music, Media, and Minds

How can a computer learn to read an ancient musical score? What can methods from signal processing and natural language analysis tell us about the history of popular music? Can a computer system teach a person to better use prosody (the musical pattern of speech) in order to become a more effective public speaker?

These are some of the questions that students will investigate in our REU: Computational Methods for Understanding Music, Media, and Minds. They will explore an exciting, interdisciplinary research area that combines machine learning, audio engineering, music theory, and cognitive science. Each student will work in a team with another student and will be mentored by two or more faculty members drawn from Computer Science, Electrical and Computer Engineering, Brain and Cognitive Science, the program in Digital Media Studies, and the Eastman School of Music.

Applying to the REU

You are eligible to apply if:

  • You are a 1st, 2nd, or 3rd year full-time student at a college or university.
  • You are a U.S. citizen or hold a green card as a permanent resident.
  • You will have completed two computer science courses or have equivalent programming experience by the start of the summer program.

It is not a requirement that you are a computer science major, or that you have prior research experience. We wish to recruit a diverse set of students, with different backgrounds and levels of experience. We encourage applications from students attending colleges that lack opportunities for research, and from students from communities underrepresented in computer science.

Before starting the application, you should prepare:

  • An unofficial college transcript, that is, a list of your college courses and grades, as a pdf, Word, or text file.  Include the courses you are currently taking.
  • Your CV or resume, as a pdf, Word, or text file.
  • A 300 word essay as a pdf, Word, or text file explaining why you wish to participate in this REU, including how it engages your interests, how the experience would support or help you define your career goals, and special skills and interests you would bring to the program.
  • The name and email address of a teacher or supervisor who can recommend you for the REU.

The application website does not allow you to save and resume your application before submitting, so start the application when you have time to enter all the information.

STEP 1: Apply online (application portal to open on December 1, 2017) no later than February 15, 2018.

STEP 2: After submitting online application, have the teacher or supervisor who can recommend you for the REU send a letter of recommendation to

STEP 3: Notification of acceptance will be communicated between March 15-April 15, 2018.

The REU Experience

The REU is 9 weeks long, running from May 29th to July 31, 2018.

Students accepted into the REU will receive:

  • On-campus housing
  • Meal stipend
  • A stipend of $4,500 for other expenses and to take back home
  • $540 to help pay for your travel to and from Rochester

Your experiences will include:

  • A programming bootcamp to help you learn or improve your programming skills in the language Python.
  • Working with a team of students and faculty on one of the projects.
  • Workshops on topics such as career planning and preparing for graduate school.
  • Social events, including a trip to the Rochester International Jazz Festival.

If you have questions about the REU or application process that are not answered here, please send an email to


You will be assigned to a project based on your background and goals.  Please note that you cannot specify a specific project on the application, but we will do our best to match you with one that matches your interests.  The projects evolve and change each year, but the projects from the past will give you an idea of the range of projects that will be available in upcoming years.

2018 Projects

Audio-Visual Scene Understanding

Mentor: Chenliang Xu (Computer Science)

Assessing the Effectiveness of a Speaker by Analyzing Prosody, Facial Expressions, and Gestures

Mentor: Ehsan Hoque (Computer Science)

Computational Methods for Social Networks and Human Mobility

Mentor: Gourab Ghoshal (Physics and Astronomy)

Computational Methods for Audio-based Noninvasive Blood Pressure Estimation

Mentor: Zeljko Ignjatovic (Electrical and Computer Engineering)

Music and the Processing Programming Language

Mentor: Sreepathi Pai (Computer Science)



2017 Projects

Click here to see student presentations from summer 2017.

Deep Learning of Musical Forms

Mentors: Daniel Gildea (Computer Science), David Temperley (Eastman School of Music). Train deep artificial neural networks to recognize motifs in classical music.

Reverse-Engineering Recorded Music

Mentors: Professors Mark Bocko and Stephen Roessner (Electrical and Computer Engineering) and Darren Mueller (Eastman School of Music). Use signal processing algorithms to discover how the same recordings were remastered over time.

Web-based Interactive Music Transcription

Mentors: Professors Zhiyao Duan (Electrical and Computer Engineering and David Temperley (Eastman School of Music). Building an interactive music transcription system that allows a user and the machine to collectively transcribe a piano performance.

The Prosody and Body Language of Effective Public Speaking

Mentors: Professors Ehsan Hoque (Computer Science), Chigusa Kurumada (Brain and Cognitive Science), and Betsy Marvin (Eastman School of Music). Measuring the visual (e.g. smiling) and auditory features (e.g. speaking rate) that cause a speaker to be highly rated by listeners.

Synthesizing Musical Performances

Mentors: Professors Chenliang Xu (Computer Science), Jiebo Luo (Computer Science) and Zhiyao Duan (Electrical and Computer Engineering). Using deep generative learning to synthesize video of a musical performer from audio input.

Reading Ancient Manuscripts

Mentors: Professors Henry Kautz (Computer Science) and Greg Heyworth (English).  Using deep artificial neural networks and probabilistic algorithms to transcribe Medieval Latin manuscripts.


2018 Participants

Erik AzzaranoRochester Institute of Technology (RIT)
Alexander BerryMiddlebury College
Adriena CribbUniversity of Pittsburgh
Nicole GatesWellesley College
Justin GoodmanUniversity of Maryland - College Park
Kowe KadomaFlorida Agricultural and Mechanical University
Shiva LakshmananCornell University
Connor LuckettAustin College
Marc MooreMississippi State University
Michael PeymanMesa Community College

2017 Participants

Jake AltabefRenssaleaer Polytechnic Institute (RPI)
Harleigh AwnerCarnegie Mellon University
Moses BugBrandeis University
Ethan ColeUniversity of Michigan
Adrian EldridgeUniversity of Rochester
Arlen FanUniversity of Rochester
Sarah FieldUniversity of Rochester
Lauren FowlerMercer University
Graham PalmerUniversity of Michigan
Astha SinghalUniversity of Maryland
Wesley SmithUniversity of Edinburgh (UK)
Andrew SmithUniversity of Central Florida