Bionic arm gives cyborg drummer superhuman skills

JASON BARNES had wanted to be a professional drummer since he was a teenager. But when he lost his arm in a freak accident he thought his dream was over. Now he has a second chance at the big time – thanks to a brand new robotic arm.

Barnes lost the lower half of his right arm two years ago, after getting an electric shock while cleaning a vent hood in a restaurant. But he refused to give up on his musical dream, so he built a simple drumming device out of a brace and some springs that attached to his arm. Although Barnes couldn’t play quite as adeptly as before the accident, he was able to enrol at theAtlanta Institute of Music and Media in Georgia, and to make plans for a future career in music.

At the institute, Barnes caught the eye of drum instructor Eric Sanders, who introduced him to Gil Weinberg at the Georgia Institute of Technology. The group hatched a plan to build Barnes a robotic arm that would allow him to play just as well as any human drummer – or perhaps even better.

Last week, Barnes tried the finished device for the first time (see video). “It was pretty awesome,” he says. “If it works out and it proves to be a lot more useful than my current prosthesis, I would definitely use it all the time.”

In the past, Weinberg’s lab has built a number of stand-alone musical robots, like robotic drummer Haile and marimba-playing Shimon. Both can listen to what human musicians are playing and improvise accompanying beats of their own.

“In some cases, we were able to create some surprises, with music that cannot be created by humans alone,” says Weinberg. “The next interesting step is to see what happens when you are part of the robot and the robot is part of you.”

For Barnes, the device needed to be able to take cues from the human body. The lab designed a prosthesis that uses a technique called electromyography to pick up on electrical signals in the upper arm muscles. By tensing his biceps, Barnes controls a small motor that changes how tightly the prosthetic arm grips the drumstick and how quickly it moves, vital skills for a drummer.

The researchers then added another layer of complexity: a second, autonomous drumstick on the robot arm (see photo). This second stick, controlled via its own motor, uses a microphone and an accelerometer to sense the rhythm Barnes is playing, as well as music from any nearby musicians. An algorithm then produces a new beat with a complementary rhythm and melody, modelled on the music of jazz greats like John Coltrane and Thelonious Monk.

With this extra artificial intelligence, human and machine combine to make Barnes a kind of “superhuman drummer”, Weinberg says.

It’s a common theme that runs through most of Weinberg’s work – the idea that robots can help us make music that wouldn’t be possible by humans alone. For example, software can crunch data much more quickly than we can, he says. It can also combine different musical styles in unexpected ways.

“This could change the way we interact with our instruments in the future,” says Jack Baker, drummer with the band Bonobo. “It’s not only a great step forward for amputees but for drummers who are willing to use technology. I would love to see it in action.”

Sanders says he could definitely imagine playing with a robot in the future. “I have a fusion trio group that plays really challenging music. It would be kind of perfect to improvise with a robot and see where that takes us,” he says.

Since percussion requires such exact timing, Weinberg’s team hopes that its algorithms will find a home in other fields that require a high level of accuracy, such as self-driving cars or assistant robotic arms for astronauts.

The prosthesis will make its debut in a concert at the Atlanta Science Festivalon 22 March. Barnes will drum alongside Sanders and some of the lab’s other creations, including a dancing robot named Shimi.

Before then, the researchers need to put a few finishing touches to the prosthesis, including fitting it more closely to the muscle signals from Barnes’s arm. And, like any other musician, Barnes needs to practise. “I have a lot of work to do,” he says.

This article appeared in print under the headline “The beat machine”