GMU:Audio+Video/projekte/MusicPainter

the pitch

I started with a series of patches isolating each note from 3 octaves. Each note was linked to one form, and amplitude made it bigger or increased its opacity. I tried diffrent sources of sound to test them, like tunes of music or my own voice. But due to the harmonics, I had difficulties to isolate this way the pitch, even with filters. For exemple, with my low pitched voice, it gave more result on quit high pitched voices, whereas higher voice could have more influence on elements linked to lower notes.

I tried again to use this method to create later an abstraction for bass instrument. But the result were quit bad too, it was difficult to get the feeling of following the music.

I also tried to use the pitch information to change the vertical position of one element. But again, due to the harmonics and the simply fact that different notes can be played by the an instrument in the same time, it didn't follow the melody very well.

That's why I decided to give less importance to pitch in my work, and tried to work more on other audio informations : amplitude and attack.

I also understood that movement (in space or by size changement) was more importante to follow the music, especially when it's linked to the rythm. So I prefered making the pitch acting more on color changes than on movement. But I had an other problem. Working mostly on particles, my diffrent absractions (one for each instrument) using particles switched their color information. Guitar's colors were sent to drums' forms, drums' colors sent to strings, etc... I tried to switch myself the color part of each abstraction following this way. But it was still unstable. For exemple if I use 5 abstractions for five audio sources on 6, it's switched again.

So I decided to use this problem to solve an other problem, the fact that colors were more or less the same for each tunes. I used the [part_targetcolor] object to get nuances in color changes. So the colors doesn't act on the good forms, but finally, as the most important is movement for me, it's not so important. The main aim is to create a global animation following music. In a musical composition, the instruments play together and create one main feeling. It's the same I do. When an instrument play higher, even if its color evolve on another form than the correct one, the watcher will notice a visual changement linked to an audio changement. Even if he can link himself wich form corresponds to each instrument, it's not the most important. The main element is the global one. To use pitch information I use the [sigmund~] object.

Amplitude and Attack

I used amplitude to incresed or decrease the size of an element. To get it, I used the [sigmung~] object or the [env~] one. I used attack information to increase the speedness of particles. For exemple with the piano's and guitare's abstractions, each attack increases suddently the speedness and makes the older particles disappear. To use it, I used the [fiddle~] object.

Instrument's Abstractions

//[Singer] [pix_kaleidoscope] object acting on a picture to choose. This picture is a spiral with horizontal lines, avaible in different colors. This is the only one abstraction working without particles. Number of segments and angle degrees : Pitch. Scale (size) : Amplitude.

//[Piano] Working with particles (vertical rectangle). The rectangle turns around the center of the window. At each attack, the old ones diappear. Radius : 1,75 Size : Amplitude. Speedness : Attack information. From 0,01 to 2. Each attack makes the particles rotate on itself of 5 degrees.

//[Guitar] This abstraction work the same way than the piano's one, but with a radius of 2,5.

//[Strings] This one can also be used for voices (choir). The particles (triangles) turn also around the middle, but the raius can change. Radius : Pitch. Size : Unic, but evenly decreasing when strings don't play anymore. Numbers : Amplitude. Each attack makes the particles rotate on itself of 10 degrees.

//[Bass] This abstraction works primary the same way than the strings' one. The differences are : - the orbit is oblique; - the speedness is lower; - the particles are circles; - the size depends of the pitch; - the particles don't turn on themselves.

//[Drums] This abstraction works with particles, but those ones don't turn around the center. Y position : Pitch (randomly between 2 and 4 or -2 and -4, it can change after each attack). X position : Randomly between -3,5 and 3,5. Size : Amplitude. Speedness : 0,01.

Player

You can download the patch here : File:MusicPainter.zip.

Result