
Shimi
Multimodel Robot Used to help people who are deaf and hard of hearing experience music
Shimon Generating different vocal and marimba parts to cover songs
This started out with lyric generation and rapping with Jeff Goldblum, then became a project presented at the Georgia tech both at Supercomputing 2024.
We used stem separation to seperate the lyrics from instrumentals. We then converted the instrumental audio to MIDI and applied a “shimonizer” to it. This allowed the notes to be played in a way that is possible based on shimons morphology.
We then got the lyrics and pitch data to generate a shimon specific voice and produced various coer songs to play at Georgia Techs Booth at SC24