The Hearing Body: How auditory perception influences body representation is a project led by Dr Ana Tajadura-Jimenez at the Interactive Centre at UCL.

After the success of the ‘Interactive Bus Stop‘  my name was put forward as an accomplished Max/MSP developer and I was asked if I would be interested in working on short-term project, coding a gesture sonification patch that would be used in an experiment as part of The Hearing Body project. After consultation with Ana, along with Frédéric Bevilacqua (IRCAM) who would also provide code, I was given a working granular synthesiser patch that used external MuBu libraries, accompanied with the following build requirements:

• Read input position of a finger, and it’s applied pressure when the finger is touching the surface of a tablet (A Wacom Intuos Pro or a Synaptics ForcePad).
• Calculate the velocity of the fingers motion.
• Load in several sounds.
• Dynamically map the provided audio samples of the different textures being touched with different force and speed to respond accordingly.
• Have the option to turn-off pressure and/or velocity response.
• Add a an audio filter (highpass, lowpass, bandpass etc).
• Record the output sound to file.
• Record all data parameters relative to the audio recordings position for: X and Y position of finger, velocity, pressure,  filter type, filter value, and sample currently read.
• Provide an easy-to-use interface.

As the Wacom uses a digital pen to capture the drawings tablet pressure (and the experiment requires the use of a finger) piezos were fitted to the underside of the tablet and plugged into an Arduino to gain a useable pressure value.

After a Week I had the patch working correctly; mapping different velocity and pressure input to their corresponding samples within the choice of texture (sandpaper, velvet or paper), as well as accurate data capture and the additional requirements of the patch. The following day tests were made with the use of a ForcePad by Synaptics, however as the device is Windows based it required the data from the device to be sent via network which resulted in lag (meaning the sound did not accurate match the gesture captured). Ultimately this led to the decision to continue using the Wacom, and the patch built.

Having previously only coded with an artist output in mind, it was very interesting to work with a different criteria and conditions to adhere to. Consequently the experience was highly enjoyable, and I hope to be involved in further projects of this nature in the future.