I'm going to be redesigning the PMC over the summer so that it is no longer built into a shirt, but rather is a separate device to be worn over a shirt. I've discovered that shirt design is inconvenient, so I'm looking forward to the redesign.
Tuesday, May 24, 2011
Latest fabrication photos
I'm going to be redesigning the PMC over the summer so that it is no longer built into a shirt, but rather is a separate device to be worn over a shirt. I've discovered that shirt design is inconvenient, so I'm looking forward to the redesign.
Monday, May 23, 2011
Code/Programming Issues
The code issues for this project fall into one of two related categories:
1. Gathering data from the stretch sensors
2. Processing the breathing sounds gathered in real-time from the stethoscope, based on the data gathered from the stretch sensors
In order to gather the data from the stretch sensors, I need to ensure an active and accurate BlueTooth connection from the computer to the LilyPad. I then need to provide information to enable SuperCollider to distinguish the serial data from the LilyPad's pins A0-A5. This will be achieved by having each pin send a short message with each serial message, that acts as an ID-tag.
The processing of the breathing sounds to create an interesting and complex spectrum of pitch, timbre, rhythm and harmony is particularly challenging because the processes used to achieve these sounds need to be as efficient, and ideally as simple as possible. Often in composing NRT tape pieces I will apply a process to a sound, create a new AIF file, and then apply a new process, or another iteration of the same process to the sound, in order to achieve a desired effect. In real-time, this iterative process is not feasible, and so I will need to find other ways to achieve a musically rich processing of sounds.
Part of what makes this even more challenging, relates to the design of the PMC, and ultimately the mapping between the human-interface and computer-controlled sound output:
1. There are very few stretch sensors (6, total) which must control a wide variety of processes.
2. Manipulating the stretch sensors in a very specific way is challenging because of the nature of breathing: the performer must breathe, and their body will move to some extent with each breath. Aside from code issues, this will create a somewhat, though not entirely unique relationship between performer and instrument: the performer must "fight" their own body to achieve specific musical results.
3. The input data from the stretch sensors is very subtle, and in order to achieve a range of data, one has to grossly exaggerate the variation of breath patterns, making the most shallow breath even more shallow, and the deepest breath even deeper.
1. Gathering data from the stretch sensors
2. Processing the breathing sounds gathered in real-time from the stethoscope, based on the data gathered from the stretch sensors
In order to gather the data from the stretch sensors, I need to ensure an active and accurate BlueTooth connection from the computer to the LilyPad. I then need to provide information to enable SuperCollider to distinguish the serial data from the LilyPad's pins A0-A5. This will be achieved by having each pin send a short message with each serial message, that acts as an ID-tag.
The processing of the breathing sounds to create an interesting and complex spectrum of pitch, timbre, rhythm and harmony is particularly challenging because the processes used to achieve these sounds need to be as efficient, and ideally as simple as possible. Often in composing NRT tape pieces I will apply a process to a sound, create a new AIF file, and then apply a new process, or another iteration of the same process to the sound, in order to achieve a desired effect. In real-time, this iterative process is not feasible, and so I will need to find other ways to achieve a musically rich processing of sounds.
Part of what makes this even more challenging, relates to the design of the PMC, and ultimately the mapping between the human-interface and computer-controlled sound output:
1. There are very few stretch sensors (6, total) which must control a wide variety of processes.
2. Manipulating the stretch sensors in a very specific way is challenging because of the nature of breathing: the performer must breathe, and their body will move to some extent with each breath. Aside from code issues, this will create a somewhat, though not entirely unique relationship between performer and instrument: the performer must "fight" their own body to achieve specific musical results.
3. The input data from the stretch sensors is very subtle, and in order to achieve a range of data, one has to grossly exaggerate the variation of breath patterns, making the most shallow breath even more shallow, and the deepest breath even deeper.
Equipment List
To capture breath-movements:
-LilyPad microcontroller and BlueTooth Silver Mate to capture data and send wirelessly to a computer.
-Stretch sensors to detect breath-related movements of the body.
-Conductive thread to connect stretch sensors, LilyPad microcontroller, etc.
-Skin-tight shirt (stretch sensors will be sewn directly onto this layer).
-Extra cloth or shirts for additional layers of PMC, in order to avoid the unintentional crossing of conductive thread.
-Additional thread and fabric paint to cover fraying ends of conductive thread, etc.
-Pre-made outer-layer shirt, to house the LilyPad and BlueTooth, and for aesthetic purposes.
To capture breathing sounds:
-Stethoscope, with tube cut short
-Small microphone
-Heat-shrink wrap & electrical tape, to connect stethoscope and microphone
To process sounds and create musical piece:
-Computer running SuperCollider, with external soundcard, hooked up to two speakers. Ultimately I may like to work with a four or eight speaker setup, and to use a mixing board, as well, but this will be in later iterations of the project.
-LilyPad microcontroller and BlueTooth Silver Mate to capture data and send wirelessly to a computer.
-Stretch sensors to detect breath-related movements of the body.
-Conductive thread to connect stretch sensors, LilyPad microcontroller, etc.
-Skin-tight shirt (stretch sensors will be sewn directly onto this layer).
-Extra cloth or shirts for additional layers of PMC, in order to avoid the unintentional crossing of conductive thread.
-Additional thread and fabric paint to cover fraying ends of conductive thread, etc.
-Pre-made outer-layer shirt, to house the LilyPad and BlueTooth, and for aesthetic purposes.
To capture breathing sounds:
-Stethoscope, with tube cut short
-Small microphone
-Heat-shrink wrap & electrical tape, to connect stethoscope and microphone
To process sounds and create musical piece:
-Computer running SuperCollider, with external soundcard, hooked up to two speakers. Ultimately I may like to work with a four or eight speaker setup, and to use a mixing board, as well, but this will be in later iterations of the project.
Research Methodology
The remaining research is going to be primarily into the development of the response and mapping of the PMC. The steps I will take (and am in the process of completing) are as follows:
1. Create an initial score of what I envision the final aural outcome of the project to be.
2. Bring sounds of breathing into SuperCollider and see what processes I can apply to the sounds and what transformations of the sounds I can achieve.
3. Modify the original score in order to take into account the results of the previous step.
4. Come up with a state diagram that determines how the data from the stretch sensors helps achieve the desired score.
5. Practice the piece, wearing the PMC, and see if the desired effect is achieved.
6. Modify the state diagram and code to improve mapping and human-computer interaction as necessary.
7. Repeat steps 5 & 6 until desired effect is achieved.
8. Practice piece with final mappings, working on enhancing subtlety of piece with breath control.
1. Create an initial score of what I envision the final aural outcome of the project to be.
2. Bring sounds of breathing into SuperCollider and see what processes I can apply to the sounds and what transformations of the sounds I can achieve.
3. Modify the original score in order to take into account the results of the previous step.
4. Come up with a state diagram that determines how the data from the stretch sensors helps achieve the desired score.
5. Practice the piece, wearing the PMC, and see if the desired effect is achieved.
6. Modify the state diagram and code to improve mapping and human-computer interaction as necessary.
7. Repeat steps 5 & 6 until desired effect is achieved.
8. Practice piece with final mappings, working on enhancing subtlety of piece with breath control.
More Fabrication Images
The following are images of the under-layers of the PMC. These show the stretch sensors with conductive thread connecting the grounds, and connecting the opposite ends through resistors, to power. These ends will also connect to A0-A5 on the LilyPad to send changes in resistance to the computer, through BlueTooth. The LilyPad is on the top-most layer of the shirt, which is not pictured here, yet.
Thursday, April 28, 2011
Mechanics/Fabrication Issues
I am in the early stages of creating a prototype of the PMC.
Initial objectives:
1. Effective sensor placement for useful breathing data.
2. Comfortable design for lengthy performances, easy on/off.
3. Minimize noise, maximize accurate data.
4. If possible, create haptic response system for performer.
5. Aesthetically pleasing and efficient design.
1. Effective sensor placement for useful breathing data.
The current design's sensor placement is based both on book-study of the respiratory system and on self-observation of breath-related body movements. As you inhale, the ribcage expands to the sides; since the right lung is slightly larger than the left lung due to the location of the heart, and is divided into three rather than two lobes, the right side of the torso is equipped with two sensors, and the left with only one. The diaphragm moves involuntarily with the breath, and a sensor is placed to capture this movement. While many technical studies of breath suggest that shoulders should not move, in observation of casual breathing it was clear that shoulders regularly move in breath patterns. Therefore, a sensor is placed to capture this vertical motion. Finally, the expansion across the upper back seemed worth capturing, so a sensor is placed between the shoulder blades.
2. Comfortable design for lengthy performances, easy on/off.
The challenge in the comfort of the design is that in order to enable the stretch sensors to gather accurate breath-movement data, at least one layer of the shirt must be skin tight. This has potential comfort-ramifications in both the physical and psychological realms, depending on the performer's comfort or lack thereof with wearing exposing clothing. In my design for the PMC, the issue of psychological comfort is addressed by creating a multi-layer design in which the top-layer is not form-fitting. The issue of physical comfort is addressed in a somewhat less satisfactory manner. The shirt has 4 layers, and will therefore be not only tight, but hot. This makes the ideal performance space somewhere that is cold, or at least cool. I will attempt to address these issues better in future revisions. Putting the shirt on, or taking it off, is not quite as easy as would be ideal, but it could be worse. I achieved a somewhat easier on/off by sewing together lots of key places in the layering of the shirts.
3. Minimize noise, maximize accurate data.
Minimizing noise is actually relatively simple in the PMC design, because both the changes in resistance provided by the stretch sensors, and the movement of the body during inhalation and exhalation, is relatively subtle. Therefore, the nature of the instrument inherently minimizes noise. Maximizing accurate data is more of an issue, due to the subtle nature of the instrument. Because of this, three of six stretch sensors (those placed on the side-body, at various places on the rib-cage), have been doubled-over in the shirt design, to double the changes in resistance with each movement. Two of the remaining three are placed in locations where it is easy for the body to exaggerate movements as necessary (the shoulder and between the shoulder blades). The remaining stretch sensor is placed on the front-body, between the bottom right and left ribcage. This stretch sensor will likely register the least amount of changes in resistance, but this is by design--this means that it can be used to trigger things only as desired, and with an exaggerated effort by the performer.
4. If possible, create haptic response system for performer.
This particular element will not be explored in this iteration of the project. Because a person can feel their own breathing, if they focus their attention, this seemed less important, and possibly redundant. The concept will be revisited in future versions, and a final decision will be made as to its usefulness and importance.
5. Aesthetically pleasing and efficient design.
The design that is visible to the audience is relatively efficient. While I find it aesthetically pleasing as well, some feedback has indicated that it may have more sci-fi/fantasy connotations than I desired. The placement of the stretch sensors made it very difficult to create an efficient path for all of the various strands of conductive thread, and therefore I settled for simply an *accurate* design. Layering helped keep the accuracy, and ended up being the most efficient course of action that I could determine.
Other Issues:
For the physical controller, I imagined that the trickiest element would be incorporating stretch sensors into the textile so that they moved; if each end of the sensors was not properly anchored, the data-change from the sensors would be limited or absent.
My initial sketches followed the following logic:
Knit fabrics are stretchy and will move with the sensor; this material should be placed underneath each stretch sensor. Woven fabrics, on the other hand, do not stretch, and make good anchors for the ends of the sensors. The woven fabrics need to also be anchored around the body, so that they do not simply move with the rest of the fabric.
In my first attempt to resolve this issue I designed a dual-layer shirt which incorporates knit fabric as the base, with a vest-like woven fabric anchoring top-layer. The hope is that the material will be firm enough, if wrapped around the entire torso, to provide the necessary anchor for the sensors.
I subsequently discovered that the anchoring was not as much of an issue as I had envisioned, and was able to use a knit tank-top as the base on which I sewed all of the stretch sensors.
Regarding the stethoscope-microphone, I was able to appropriate a cloth belt, on which I affixed the stethoscope, and in this manner I can attach the mic around the neck of the performer.
Initial objectives:
1. Effective sensor placement for useful breathing data.
2. Comfortable design for lengthy performances, easy on/off.
3. Minimize noise, maximize accurate data.
4. If possible, create haptic response system for performer.
5. Aesthetically pleasing and efficient design.
1. Effective sensor placement for useful breathing data.
The current design's sensor placement is based both on book-study of the respiratory system and on self-observation of breath-related body movements. As you inhale, the ribcage expands to the sides; since the right lung is slightly larger than the left lung due to the location of the heart, and is divided into three rather than two lobes, the right side of the torso is equipped with two sensors, and the left with only one. The diaphragm moves involuntarily with the breath, and a sensor is placed to capture this movement. While many technical studies of breath suggest that shoulders should not move, in observation of casual breathing it was clear that shoulders regularly move in breath patterns. Therefore, a sensor is placed to capture this vertical motion. Finally, the expansion across the upper back seemed worth capturing, so a sensor is placed between the shoulder blades.
2. Comfortable design for lengthy performances, easy on/off.
The challenge in the comfort of the design is that in order to enable the stretch sensors to gather accurate breath-movement data, at least one layer of the shirt must be skin tight. This has potential comfort-ramifications in both the physical and psychological realms, depending on the performer's comfort or lack thereof with wearing exposing clothing. In my design for the PMC, the issue of psychological comfort is addressed by creating a multi-layer design in which the top-layer is not form-fitting. The issue of physical comfort is addressed in a somewhat less satisfactory manner. The shirt has 4 layers, and will therefore be not only tight, but hot. This makes the ideal performance space somewhere that is cold, or at least cool. I will attempt to address these issues better in future revisions. Putting the shirt on, or taking it off, is not quite as easy as would be ideal, but it could be worse. I achieved a somewhat easier on/off by sewing together lots of key places in the layering of the shirts.
3. Minimize noise, maximize accurate data.
Minimizing noise is actually relatively simple in the PMC design, because both the changes in resistance provided by the stretch sensors, and the movement of the body during inhalation and exhalation, is relatively subtle. Therefore, the nature of the instrument inherently minimizes noise. Maximizing accurate data is more of an issue, due to the subtle nature of the instrument. Because of this, three of six stretch sensors (those placed on the side-body, at various places on the rib-cage), have been doubled-over in the shirt design, to double the changes in resistance with each movement. Two of the remaining three are placed in locations where it is easy for the body to exaggerate movements as necessary (the shoulder and between the shoulder blades). The remaining stretch sensor is placed on the front-body, between the bottom right and left ribcage. This stretch sensor will likely register the least amount of changes in resistance, but this is by design--this means that it can be used to trigger things only as desired, and with an exaggerated effort by the performer.
4. If possible, create haptic response system for performer.
This particular element will not be explored in this iteration of the project. Because a person can feel their own breathing, if they focus their attention, this seemed less important, and possibly redundant. The concept will be revisited in future versions, and a final decision will be made as to its usefulness and importance.
5. Aesthetically pleasing and efficient design.
The design that is visible to the audience is relatively efficient. While I find it aesthetically pleasing as well, some feedback has indicated that it may have more sci-fi/fantasy connotations than I desired. The placement of the stretch sensors made it very difficult to create an efficient path for all of the various strands of conductive thread, and therefore I settled for simply an *accurate* design. Layering helped keep the accuracy, and ended up being the most efficient course of action that I could determine.
Other Issues:
For the physical controller, I imagined that the trickiest element would be incorporating stretch sensors into the textile so that they moved; if each end of the sensors was not properly anchored, the data-change from the sensors would be limited or absent.
My initial sketches followed the following logic:
Knit fabrics are stretchy and will move with the sensor; this material should be placed underneath each stretch sensor. Woven fabrics, on the other hand, do not stretch, and make good anchors for the ends of the sensors. The woven fabrics need to also be anchored around the body, so that they do not simply move with the rest of the fabric.
In my first attempt to resolve this issue I designed a dual-layer shirt which incorporates knit fabric as the base, with a vest-like woven fabric anchoring top-layer. The hope is that the material will be firm enough, if wrapped around the entire torso, to provide the necessary anchor for the sensors.
I subsequently discovered that the anchoring was not as much of an issue as I had envisioned, and was able to use a knit tank-top as the base on which I sewed all of the stretch sensors.
Regarding the stethoscope-microphone, I was able to appropriate a cloth belt, on which I affixed the stethoscope, and in this manner I can attach the mic around the neck of the performer.
Subscribe to:
Posts (Atom)