Musical Prosthetics

Detail: Our original idea

Jasper Sims

While researching and learning about musical prosthetics, we saw a video about a project that was made called the Dodecahedron.  This is an instrument that is controlled by the movement of your hands.  It plays different sounds depending on which side of the instrument you play.  Our group formed based on our interest in this video, and we created our first sketch of a possible prototype.  The original idea was to make 3 triangular panels that had infrared sensors that would detect the movement of our hands.  We planned on controlling the volume through the distance of our hands to the sensors.  We also wanted to include LED lights to add more of an effect to the prosthetic.  We were hoping the panels would be able to spin, and based on the speed of the rotation, the light would flicker different colors.  After thinking through all these aspects we wanted to include, we came up with the name “Merkaba.” Merkaba is a form of meditation that uses one’s energy, directing it across three triangles surrounding the person.  We liked the similarity in focusing on triangles, so we decided to name our project that.  We spent a few days sketching the triangular panels and discussing how to build the prosthetic.  However, after further deliberation, we decided it would be better to make the musical prosthetic be something one can wear.  This brought us to our final project, which was a bracelet.  We faced many more changes after we thought of the bracelet, but we had narrowed down our brainstorming ideas, and were ready to start on the technical and design piece.


Lizzie Beer


Vocal Bracelet (Isabella):

The vocal bracelet process started with assembling the melody and the beat to go together in the software Ableton Live. Then Isabella was able to record her own vocals with one of the coaches. This took a couple takes and a several tests to determine which affects worked out. After recording Isabella’s vocals we decided to add one of the coaches awesome beatboxing skills. After recording both of Isabella’s and her coaches vocal samples we put them into Ableton Live and edited them. This consisted of elongating some parts and minimizing others. This helped because it escalated parts of the track such as the piano and beat. After creating the track, Isabella cut out main samples and made them into loops. A factor Isabella had to consider during this process was to have it be eight bars and work together when her group performed live. My next step was to send each loop to my partners and all of us had to set up the sensors on max patch. However our first attempt to do so was unsuccessful because we needed to fix the patch for each individual. After doing so we connected all the wires and placed the sensors on.

Melody Bracelet (Jasper):

For the melody bracelet Jasper downloaded a package of 300 synth, piano, and bass samples onto itunes. He sorted through them until finding a moody and catchy piano riff that the whole group agreed on. This piano riff was the basis for which the vocals and beatboxing were created - all the other recordings (except the beat tracks) were recorded over the piano sample. After Isabella sung the vocal track, One of the coaches took all the samples and recordings to Ableton Live, changed all the BPM’s (beats per minute) to rhythmically align everything, and added a bass track of his own creation to pull it all together. This bass track also ended up on Jasper’s bracelet along with the piano tune.

Beat Bracelet (Lizzie):

The beat bracelet process started with finding loops online that would correspond well with the melody.  Lizzie found multiple loops that sounded sufficient, however she narrowed down to two beats that would work well with the vocals and the synthesizer.  Those two beats were edited by Isabella in Ableton Live for the best quality.  After she was done editing the beats they were then downloaded to the sensors, using the Arduino.  

Vocal Track (Isabella):

Isabella created the full track by using the software Ableton Live. Isabella took the loops from the melody,beat and vocal and placed them in the software. From there she manipulated the vocals by adding and correcting each of them with different effects. Then Isabella took all the sample loops from the vocals and beatboxing and tried to see which sounded best together. Each sample Isabella cut she had to manipulate in order to make fuller sounds. This meant clicking each individual sample and either speeding up the song or elongating them. The final step consisted of rasterizing the whole piece in order to get a final song.


Bracelet Design (Lizzie):

After settling on the bracelet idea, we still had many more aspects of the project to figure out.  Lizzie started by working on rhino.  She didn’t know exactly what the bracelet would finally look like, so she played with some of the tools on rhino, and came up with a couple different ideas.  After a while of designing different kinds of bracelets on rhino, Lizzie decided to research some exotic looking bracelets, for some inspiration. She explored different textures, designs and themes.  After collecting multiple images of different artist’s work, Lizzie started sorting through materials and found cardboard.  She started designing strips in rhino and laser cut them into the cardboard.  To give it a minor effect, she made the strips uneven.  Some were bigger than others.  After gathering the strips from the laser cutter Lizzie wove them together.  This was one option for the texture of the bracelet.  To make the woven pattern look more exciting, Lizzie laser cut arcs out of cardboard, and glued them in between the strips.  It captured the creative look we wanted, but it was not functional as a musical prosthetic.  

This is when Lizzie started with constructing a whole new look, that was inspired by a wooden bracelet she found online.  She was planning on laser cutting the pieces, so she started designing it on rhino.  It was cut into foamboard.  This idea was using 4 different pieces of the bracelet and connecting them by wooden dowel.  After the first prototype of this idea was laser cut, Lizzie fit them onto her wrist, and realized the edges were too sharp for comfort, and she had to readjust the size.  Lizzie applied these critiques to the design in rhino, and laser cut the second prototype.  This one worked and fit well onto our wrists.  However, after Lizzie started to use the dowels, she realized the only size that we had access to, were a little smaller than the ones she had made room for.  Given this challenge, she worked on the design in rhino some more, and laser cut the final prototype.  She fit the dowels in, and put the bracelet together.  

After more discussion about the bracelet, we came to the conclusion that touch sensors would offer more control over the sounds than the distance sensors.  This meant we had change the design of the bracelet to fit the touch sensors.  Lizzie started 3d modeling on rhino.  She used the dimensions she had previously gathered and created multiple bracelets with different features.  Each one had a different look.  Some were sharper, or thicker, or longer.  She examined each design and considered the functionality each bracelet.  Lizzie finally had the basic design of the bracelet, with the correct dimensions, and thickness.  However, this design looked quite boring to us, and did not fulfill the captivating aspect that we were hoping it would.  

This is when Lizzie started brainstorming possible additions the bracelet could include.  This is what took her to the final bracelet.  She added texture to one side of the bracelet, by added spikes.  It captured the look we thrived for, and we were satisfied with the functionality of the musical prosthetic piece.  

Final - Product

Isabella Julian

Merkaba is a 3d printed exotic looking bracelet that worked as a musical prosthetic.  The bracelet itself has sharp points sticking out on one side and extends down the bracelet, unevenly. The bracelet has an arduino attached to the bracelet for the final presentation, and multiple wires connecting the bracelet to the patch on a users arm.  The wires are soldered and placed in the correct holes of the arduino to send the data to the sensors. Each bracelet has two touch sensors, which play two different sounds according to the instrument each player is assigned.

In our team Isabella had the vocal bracelet - one of the sensors on the vocal bracelet manipulates the volume of her recorded singing , while the second sensor manipulates one of her coaches beat-boxing recordings. Jasper had the melody bracelet- both sensors on the bracelet are piano based sounds and has the manipulations of the volume. Lizzie had the beat bracelet- both sensors on the bracelet plays two different beats and has the manipulations of the volume aswell the other two bracelets.


Ellis Cordaro

The purpose of the dynamic drum trigger is for the user to be able to essentially wear a drum kit that was able to mimic the function of an acoustic drum set in that it was able to change pitch, and can be tuned to any note that the user would choose. This happens though sensors which detect the impact of the strike of the drum stick. The change in resistance goes to an Arduino, a microprocessor, which has an Xbee that transmits the readings wirelessly. The readings then go through Max MSP, a graphical programming environment, where a certain spectrum of the reading gets matched with a drum sample, thus triggering the drum sample to play through the laptop speaker.

By the end, there were 6 drum triggers (1 snare, 1 ride, 1 tom, 1 hi-hat, 1 hi-hat pedal and 1 bass drum). The snare, ride, hi-hat, and tom had piezo pickups which measured the impact of the drum stick on the head on the trigger. In the middle of these triggers there is a piece of copper attached to a capacitive touch sensor which when touched changes the fundamental note of the drum sample when triggered. The hi-hat pedal and bass drum used touch sensors that changes resistance when stepped on. The triggers themselves are made from linoleum and has an HDPE head covering them where the stick would strike. The triggers are also mounted to a larger piece of HDPE which would be strapped to the user's leg when played.  



Ellis Cordaro

My original idea for the electronic drum triggers used piezo pickups and allowed the user to set each drum to a specific note and alter the note by using some type of sensor, potentially a touch sensor. The first sketch I made had 11 triggers in total and would be as large as 20" in diameter and were individual pads that could rest on a table or on an acoustic set. I soon realized that 11 triggers would be an unreasonable task for the two weeks I had to complete the project. So, I decided to only include a 4 piece set (snare, 2 toms, bass drum) and also include a hi-hat and a ride. I then made a prototype of a trigger using 3 layers of foam core. The first layer was a backing, the second had space to house wires and a piezo pickup, the third had a space for a touch sensor (preferably analog). When we pitched our ideas to Saeed and our coaches, they recommended that the instrument be attached to the users body. While planning to create a piece of hardware that could attach to the user, I decided to scale the size of the pads down to 5" for the snare and ride, and 3" for the hi-hat and tom.

From the beginning, I knew that I wanted to use some sort of rubber as the head of the drum. The challenge was finding a conventionally available type of rubber that was responsive enough and would give enough fo a rebound to be a drum trigger. I decided to use linoleum as the head of the drum. After making 4 prototypes out of paper, I decided to lazer cut the final version of the linoleum (for the heads) and HDPE (for the part that attaches to the user's leg. I then use epoxy to connect the HDPE to the linoleum. After days of waiting for the epoxy to dry, it never lost it's sticky and gooey consistency. I was later told by David that the reason for the epoxy not curing was because HDPE is designed to prevent liquids from sticking to it. That is why the material is used for milk jugs, so the proteins in the milk don't condense on the inside of the container. However, I learned this right before the day of the presentation so it was too late for me to make any changes. Another problem I had was, while making a hole for the piezo to fit in, was I drilled right through the linoleum because it was so soft. This happened even while applying minimal pressure to the drill. So, I just decided to drill through all of the heads. I then printed acrylic heads to cover the holes. The piezos would now be glued onto the the side of the hole so that the sensitive portion of the pickup would be able to move freely and touch the backside of the acrylic.

Another time consuming aspect to my project was wiring all of the sensors to the Arduino. At this point, I had settled on all of my sensors, 4 piezo pickups (for snare, tom, hi-hat top, ride), 1 cap. touch sensor (to change pitch) and two touch sensors (for bass drum and hi-hat pedal). When the piezos arrived at the NuVu studio, I spent most of a day soldering the wires to the sensors. One thing I learned while wiring was to not use heat shrink as bundling. Using the heat shrink for bundling made it difficult to find the location that the wires ran to and create more work for me after finishing the wiring. Finally, After finishing the wiring, I was able to sew spandex straps with velcro. I also put the Arduino and wiring into a wood case I made on the laser cutter. I also epoxied the 9V battery pack onto the outside of the box. However, the final product did not work because I was not able to get adequate sensors that could get a consistent reading when triggered. Although the sensors did not output consistent readings, the programming on Max MSP did work properly.

Final - Audio

Saba Ghole

Merkaba Sound Elements

Saba Ghole


Kate Reed

We started out with the concept of an accordion corset. We thought that the accordion and a human back are similar. We then began to try and sketch out what we thought that an accordion corset would look like. It was hard in the beginning because we weren’t quite getting our idea clear enough for people to understand what we were thinking. We made a lot of sketches.

One of the first problems we came up with was what would we make it out of? We originally thought of making it out of acrylic. We soon ditched that idea because it was just way too complicated. We thought of making it out of fabric but that just wasn’t giving us the clean classy look we had in our heads. We thought about making similar to a cage skirt, with big strips of fabric and openness.

We heavily pursued origami. We thought the 3-D origami had more give and flexibility than the other materials that we were looking at, such as fabric and plastic. We tried a lot of different 3D origami patterns trying to get a feel for it and experiment how they moved.

Silly us, to think that we had found our medium so soon! We quickly realized that although the origami had a lot of give in one direction, we needed give in both directions. If we only wanted the corset to bend forward it would have been perfect, but because we wanted the corset to bend both forward and sideways so the origami wasn’t going to work.

We then decided to try a more industrial look. We experimented with hinges with small pieces connecting them. That particular set up would not work because it would be crazy to have that many hinges. While we were trying to problem solve we thought of having only two pieces of wood, but that would not work because then you would have to bend at a perfect right angle which isn’t natural. We then thought, what if the hinges stuck out? Then you would be able to bend without it digging into your side. We were on to something with the extruded hinges.

We then cut a test out of foam core, to prove the concept. We cut a thin rectangle with a small hole in it for the screw, and what do you know, it worked! We then tried to 3D model everything. We tried to think of different shapes that the corset could take on. Each arm didn’t have to be the same shape.

We decided that we thought the non-symmetrical shape was more interesting than the symmetrical shape. We also liked the arms to be curved and organic. We thought that the arms would look super cool if they had rubber band strung through them- and it did look super awesome, but it wasn’t practical. Because we had so many rubber bands, they squeezed the arms shut. They made it so you couldn’t really bend or move. But they looked super cool.

What was stressful about this studio is we didn’t really have much to physically show up until the last day. We had been doing so much on the computer whether it be programming or modeling, that the corset didn’t fall into place until the very end of the studio.

Once we did start putting it together there was a lot of excitement and busy work. Each arm had to be paired up with its partnered length and screwed together. Because we used lock bolts it took much longer to screw the arms together. Although lock bolts are awesome because they don’t fall off, they are harder to put on. They require ten pounds of force to screw in, meaning you have to hold the bolt with pliers and screw it with a screwdriver on the other end.

We also had to 3D print 16 hinge pieces, which is where the arms attach to the bottom and top belt. The belts were tricky to design, because they had to have laser cut holes that were carefully measured to fit with the 3D printed hinges we designed. The belt took awhile because every time we changed the arm design we had to change the hinge design and the belt design. We changed the design a lot!

Once the arms and belt were in place we tried putting the electronics on it. The arduino, which is a mini computer, was sewn onto the bottom belt, and the flex sensors were taped to a side arm the front arm.

Our final project looked really awesome, and it sounded really awesome. I think we were all really proud an excited about it in the end.

Body Accordion

Kate Reed

We created a corset that acts like an accordion. The corset has two belts, one at the waist and one above the bust. In between these two belts are eight arms. The arms are an organic rib-like shape that is laser cut out of plywood. Each arm is made of two pieces, which are connected together using a screw and a lock nut. We had to use lock nuts because normal nuts would fall off with the amount of usage of the arms. The arms connect to a 3D printed hinge that connects to the belt. The hinge allows the arms to move and be stable to the belt at the same time.

There are two flex sensors on the corset, one in the front and one on the side. They are positioned at the bottom of the arms and sense when you are bending forward or sideways. The flex sensors give a number based on how bent they are. That number gets sent to the computer and into a program called Max MSP. We programmed Max to change the pitch based on how bent the flex sensor is.

Musical Prosthetics

Amro Arida

For hundreds of years dancers and musicians have captivated our senses and told elaborate stories through movement and sound.  New technologies and contemporary improvisational and interactive theater are changing the way audiences experience both musical and dance performances.  These new mixed-performance experiences merge physical instruments with the human form, bringing together previously separate genres of music composition, choreography, dance and performance-art.

In this studio, students will explore new techniques and concepts for wearable musical instruments or “musical prosthetics.”  Students will create and develop these high-tech musical prosthetics, and design agile and flexible interfaces that are suitable for dancer-performers-musicians. The students' inventions will deviate from mechanical sound generation used in conventional instruments, and will instead utilize high-tech sensor interfaces and digital music.