Autor: Jeong Eun Lee

Source

Naoyuki Houri, Hiroyuki Arita, Yutaka Sakaguchi. 2011. Audiolizing body movement: its concept and application to motor skill learning. In Proceeding AH ’11 Proceedings of the 2nd Augmented Human International Conference Article No. 13 Tokyo, Japan — March 13 – 13

Summary

This article deals with a project that transforms the posture/movement of the human body or human controlled tools into acoustic signals and feeds them back to the users in a real-time manner.

The author says that sound effect plays a very big role in our common lives and events: many artificial system such as Video game, Cell Phones, etc, have been proposed for displaying information through auditory channel. (it is common to add sound effect for enhancing realistic sensation, or to give a stronger image and signs through auditory events, which let our brain learn their correlation better).

However, body information such as posture, movement and muscle forces are insensible. And that is why audiolizing these gestures can be effective to enhance and sense our movement better. It is also another method to compare the body states of different individuals or identical person with different occasions.

Practice

1. Asistance of Soldering Work (audiolization system measure temperature)
2. Audiolization of Calligraphy (a 6-axis force torque sensor attached to a brush)
3. Pole Balancing Game (3D posture sensor)
4. Acoustic Frisbee (3D acceleration sensor)

       

 

Relevance

If we want to work with different sensors as feedbacks system to understand our gesture better (signal and sign of our gestures and movements), this article might be helpful to see how they used different sensors within different subjects and occasions. We can use it as inspirations.

Source

MOCO ’15, August 14 – 15, 2015, Vancouver, BC, CanadaCopyright is held by the owner/author(s). Publication rights licensed to ACM.

Summary

This project introduces RAKS system (Remote electroAcuoustic Kinesthetic Sensing) that has been played through a belly dancer.

Sensor technologies translate the internal experiences to external, which means this system will be integrating movements from the dancer (kinetic) and sonic elements together by a wearable wireless sensor that is specifically designed for belly dance movement. In this project, LilyPad Aruduino, ADXL345Accelrerometer, Flex Sensor, and LED Rings are used. The Mapping Strategies are modeled on the relationship between playing techniques and acoustic instruments.

Translations of major movements from the dancer to the instruments:

1. Contraction and Release: Bow Pressure
2. Curving and Straightening: Modulating Waveshape
3. Accelerating and Decelerating: Pulses to Pitch
4. Movement and Stillness: Oscillators

 

Like it says, the electro music is played only depending on the movement of the bally dancer. Instead of composing electro music / sound on computer, the dancer makes each movements throughout her torso, chest, hip in order to create the sound of its own. While listening to the music, we can see how each movement of the dancer affects different sound and speed. Normally in a stage or dance performance, dancers create the movement throughout how the music is played. But in this project, we can see how dancers and music become one and integrated as a one piece at the same time.

 

Relevance

The fact that the article deals with the dance movement and its interaction between sensors on the body to the audio system, we can relevant this project to our own and have it as an inspiration in case one of our teams wants to work on RAKS system.

 

 

Source

Jan C. Schacher. 2016. Moving Music: Exploring Movement-to-Sound Relationships. In Proceeding MOCO ’16 Proceedings of the 3rd International Symposium on Movement and Computing Article No. 19. Thessaloniki, GA, Greece

Summary

“Moving Music” is focusing on learning how ‘Gesture’ can be used when working with sound, and real time generated music, and also how ‘Gesture’ influences the perception, affect and impact of music.

So it is important to learn inter-relationship and dependency between a musician and a dancer, between movement and sound through the use of electronic sound processes that are linked by technically sensing movements.

** dancer – movement – musician – sound = audience perceptions

In order to proceed this project, the dancer is equipped with two bracelet (wireless motion sensors) that capture degrees of Acceleration Rotation Attitude. Each sensor is attached to the each ankle (left, right) and one on the stomach and dancers perform and each gesture of the dancer shapes its own music.

To obtain position information of the dancer in space, the stage is observed by a depth-camera located in front center of the stage at floor level. The use of stage is implemented using a map zones overlaid on the stage. Each zone has a radial sensitivity curve that rises from its edge to the center.

 

Video: http://mgm.zhdk.ch/?page_id=1406

Relevance

I haven’t mentioned everything here, but this article deals with deeper psychological and technical ways to understand how performance on the stage to unite  the movements (dancers), music (musicians) and audience.

Since our project is about stage performance and interaction this article might be helpful to see how this project dealt with the depth-camera for the performance on the stage.