Shimon: an interactive improvisational robotic marimba player
Comments:
Felipe Othick
Cindy Skach
Reference:
Hoffman, Guy and Weinberg, Gil. Shimon: an interactive improvisational robotic marimba player. CHI EA '10. Georgia Institute of Technolog, Atlanta, GA, USA. http://delivery.acm.org/10.1145/1760000/1753925/p3097-hoffman.pdf?key1=1753925&key2=7636156921&coll=DL&dl=ACM&CFID=7443758&CFTOKEN=14060305
Summary:
Hoffman and Weinberg catalog the results of their jazz-playing improvisation robot which plays the marimba. The robot is to be accompanied by a human pianist and follows in unison the playing of the pianist. It recognizes chords being played and appropriately plays beats, rhythms and notes that accompany and compliment the pianist.In order to do this, the robot recognizes the tempo, chords and intensity of the music and plays appropriately. It has three different modules in which it can follow: call and response, opportunistic overlay improvisation and rhythmic phrase-matching improvisation. The three modules represent different ways in which the robot will respond to the pianist with simple chord progressions, anticipated melody and history-based improvisation respectively.
There is a robotic head consisting of a camera atop the structure that is intended to give the robot a more lively feel. Future work includes face recognition for this camera as well as improved methods of having the head follow the patterns of the robot.Discussion:
This article initially interested me because it mentioned a marimba. I was a percussionist in high school and often used a marimba and am familiar with its sound and capabilities. Before reading I was thinking whether or not it could make up music that sounds good. I am not good at improvisation on anything other than a snare, so I would be thoroughly impressed.
It turns out that the robot is improvisational. The article did not mention whether the robot digitally recognized the pianists music or did it through audible analog.
I watched a highlight of ht performance of this robot found here http://www.youtube.com/watch?v=jqcoDECGde8. It was amazing! The last and highest degree of improvisation was incredible. It was really interesting how the unit would be inspired by the pianist and the pianist inspired by the robot.
When I looked at a few videos at youtube, I didn't notice this one, but I think it's the best I've seen so thanks for the link! In some ways, I think this is one of the best examples of computer-human interaction I've seen because of the number of ways in which Shimon interacts with people.
ReplyDeleteI wonder if later one facial recognition could play a more important role. Shimon could improvise based on a player or the audience.
ReplyDelete