239
edits
No edit summary |
|||
Line 94: | Line 94: | ||
==Artistic approach== | ==Artistic approach== | ||
The aim was to create sounds to the microscopic pictures that are distinct and reflect the processed video input. The first step was to find parameters to extract directly from the picture and to transform them info sound in a logical way. To achieve the transformation into sound this the visual programming language Pure Data and the software Mainstage can be used. | |||
The | |||
Parameters such as | |||
*brightness | |||
*contrast | |||
*histogram | |||
*movement | |||
*colour | |||
were considered. | |||
<videoflash type=vimeo> | There were however some restrictions because of the quality of the image. The idea of using colour for example had to be dismissed because of the automatic white balance of the camera. | ||
<videoflash type=vimeo> | |||
In the end three different parameters were chosen: the overall brightness of the image, the histogram of the image, which changes over the period of a video and the movement of individuals inside the picture. | |||
===Brightness=== | |||
The parameter brightness was used to create an atmospheric sound that does not undergo any sudden changes. Long lasting notes with echo were chosen to create a general atmosphere. | |||
<videoflash type=vimeo>62855963|520|290</videoflash> | |||
The software takes the RGB values of the picture, adds them and divides them by 3 which gives the general brightness of the picture. This value is transformed to a midi compatible value to create a sound using a synthesizer software. | |||
===Histogram=== | |||
The values of the histogram of the pictures are taken and transformed into notes. This sequence is replayed over and over again. Just by listening to the histogram sound it is possible to tell if the picture has a consistent brightness distribution of if there are outliers which represent as spikes in the histogram. | |||
The histogram sound constributes to the general atmosphere by adding a sense of strangeness and oddness. | |||
<videoflash type=vimeo>62856047|520|290</videoflash> | |||
====Combination==== | |||
The following examples combine the parameters brightness and histogram. | |||
Video stills: | |||
<videoflash type=vimeo>56874082|520|290</videoflash> | <videoflash type=vimeo>56874082|520|290</videoflash> | ||
<videoflash type=vimeo>56874080|520|290</videoflash> | <videoflash type=vimeo>56874080|520|290</videoflash> | ||
Video: | |||
<videoflash type=vimeo>56874349|520|290</videoflash> | |||
===Movement=== | |||
By defining whether there is life in the picture or not, movement is a very important parameter to create and authentic sound but also a very tricky one. | |||
The aim was to provide the possibilty to tell something about the intensity and direction of the movement just by listening to the sound. | |||
This video only shows the movement parameter: | |||
<videoflash type=vimeo>62856465|520|290</videoflash> | |||
The y coordinate is the parameter which controls the pitch of the tine, that means, if the creature moves up the pitch rises, if the creature moves down the pitch also falls. | |||
The x coordinate controls how fast the notes are played. If the creature moves to the left the sound is calmer and has a low velocity, the more right the creature moves, the faster the sound gets. | |||
There are certain limitations. The detection is not completely accurate, the cross hair jumps up and down within the creature. It is however possible to tell something about the kind of movement that is performed in the picture. | |||
====Combination==== | |||
The following example combines all three parameters: | |||
<videoflash type=vimeo> 62856497|520|290</videoflash> | |||
The tracking works well for videos with only one moving object. The problem of jumping cross hairs is increased when there are more moving objects but although the sound does not reflect the direction of movement anymore, it reflects the intensity of movement. | |||
<videoflash type=vimeo> 62856099|520|290</videoflash> | |||
===End Result=== | |||
By combining all three parameters a sound atmosphere is generated that is directly linked to the images seen. Altogether it creates a feeling of a microscopic word that is, although it is part of our every day life, a world completely foreign to us. The foreignness already immanent in the picture is intensified through the sound. | |||
= | Furthermore the approach can be seen as a possibility to perceive something within the picture that is not directly visible to the eye. A slight change in the brightness of the picture for example can be heard much easier than seen. This way a further development of the approach could be used to help perceiving microscopic images, possibly even in a scientific way. | ||
<videoflash type=vimeo> 62856510|520|290</videoflash> |
edits