Safe And Sound

Safe and Sound is a project I helped write an NSF funded grant for! In this project, we are using music to improve safety and fluency. Currently, modern robot sonification consists of repetitive beeps (think of a truck backing up) that can become annoying to listen to for  longer than a few seconds. Hospitals have similar issues where they have many beeps that demand attention all the time (even when they don’t need it) which can cause healthcare workers to grow anxious or even tone out sounds (called alarm fatigue).

Conversely, people can listen to music for hours and extract a lot of information without growing bored or becoming desensitized. If you look at the lofi girl youtube channel, people listen to it for hours on end. Our goal is generate songs that can modify specific musical features ( such as tension and rhythmic stability) to communicate the danger and urgency level of a robot, without constantly demanding our attention.

This does mean that the sound will become urgent when the robot will crash, but if the robot is moving heavy payloads, its good to know the danger but no one needs to shout it at me. In other words,  I can listen to my lofi robot music:

If I wanna know the status of my robot I pay attention to the music a little more, but if I wanna focus on my work, the robots music will not distract me. If the robot requires my focus,  it will  change the rhythm in a way to catch my attention.

We first gathered data of music clips to understand the relationship between higher level musical features and safety and urgency. Based on this we came up with a framework to generate safe audio

Framework to sonify robots with high level music features

To improve fluency and coordination between users and robot, I’m leveraging predictable elements of music for  humans and robots to sync their task to (for example, the pop goes the weasel melody, or if you are a Bostonian everyone knows when to shout ““BUM BUM BUM” after “sweeeeeet caaaaaroliiiine”). We can predict when the last BUM of Sweet Caroline is and then hand an object to the robot. Lastly, Since I love hardware, Im also working on a cool new way to present this audio to users, so stay tuned for that!

Overview on fluency to improve synchronization

We are also working on some cool hardware applications to embed the audio of this project!

Relevant Publications

What Sounds Dangerous? Establishing Correlations Of Musical Features and Perceived Safety in HRI
Amit Rogel, Jack Haley, Richard Savery, Gil Weinberg
2025 HRI Conference

Do Re Mi Fa So Pass the tool? Using Melodic Prediction to Improve Human-Robot Fluency
Amit Rogel, Jack Haley, Gil Weinberg
2025 RoMan Conference