GROUP WORK BY
— Shih Li Chao (soundfile composition LogicPro + Max/MSP stuff)
— Sebastian Richter (idea development, tried to solve problems + good soul of the group ;))
— Adam Streicher (Unity implementation)
— Jessica Hüttig (Processing input sketch + wiki documentation)
Concept
Our idea was to use the mouse input as a medium for selecting and pitching sounds generated by Logic Pro via Max/MSP.
Additionally we wanted to use Unity 3D for the visualization of the manipulated sounds.
Diagram
HINT
Processing – sending port: 7474
Max/MSP – receiving port: 7474 and its sending port to Unity: 7475
Unity – receiving port: 7475
Implementation
Our sketch is divided in four color areas and sounds. Each sound is related to one area and each color area expresses a different range of emotions:
- red for passion and rhythm
- yellow for brightness
- green for nature and peace
- blue for mediation
Input
In Processing we divided the black sketch screen in four areas. We didn't colored the four fields in the sketch, so the experience for the user more exciting and not that obvious. It's a kind of searching for sounds... Take a look at the code here.
As a basis for the OSC connection, we used the sketch "oscp5sendreceive" by Andreas Schlegel and changed it to our needs.
We also added some functions to send data to Max/MSP: the active area, the mouse speed and the mouse position.
Max/MSP is receiving the data from Processing and we use it to interact with the sound.
The composition of the sound-files happened in LogicPro. Max/MSP is used to add and control the sound effects.
Audio Output
In these two captures you can see that we used the four different color areas to map different sounds. Each of them has its own sound effect.
Red Area – Passion
For this part, we used the scrub~ function to generate a delay effect with a changeable ramp and resample rate. The speed of the mouse is controlling the delay time, ramp and volume.
The faster you move the mouse, the more delay effect you will hear.
Yellow Area – Brightness
In this case, we used additive synthesis and ring modulation for the sound effect. That way we could get more different frequency rates into the original sound to generate a brighter timbre. The user can move mouse to change the modulation wave and use different mouse speeds to change the ring modulation.
Green Area – Nature
We thought pitch shifting is the most adaptive way for nature sounds, because it could keep the original figure of sound without too much deforming. However, you can still hear the variety within that. The faster the speed of mouse movement is the sharper voice will become.
Blue Area – Mediation
This song conveys a sense of peace and mediation. Because of that, we decided to use the reverb effect to generate a wide diffusion of sound in space. For example you can create a sound-scape that involves people. When you change the mouse speed you will get a different sound diffusion.
BONUS – Surround Sound System
Using pi and some expressions we generated a four channel sound-system. When you click the toggle, it will trigger the automatic 3D sound system effect with a circular path.
Visual Output
The next step is the connection from Max/MSP to Unity via OSC. Our original idea was to create an equalizer with the help of a script from Unity, but it was created with an audio-input file in mind. This means, you had to import a WAV-File in Unity to obtain nice results.
We tried to rewrite the script, but because of the harsh time constraints and not that advanced coding skills, we weren't able to manage it. We had problems to connect and receive the messages from Max/MSP in Unity. But we found an alternative OSC-receiver skript, that allows us to receive messages via OSC. We rewrote it a bit, so that a blue cube is rotating around its z-axis. The rotating depends on the mouse speed. The faster you move the mouse, the faster the cube is rotating. As you can see in the screenshots, there are still some problems to solve, but at least all three computers are connected and receiving OSC messages from each other!
So we are happy to report that we solved the task. :)
