PDCON:Conference/A Pd framework for the Xth Sense: enabling computers to sense human kinetic behaviour: Difference between revisions

Latest revision as of 15:31, 20 February 2012

A Pd framework for the Xth Sense: enabling computers to sense human kinetic behaviour

Author: Marco Donnarumma

Download full paper: Media:A Pd framework for the Xth Sense-enabling computers to sense human kinetic behaviour.pdf

The Xth Sense is an interactive system for the biophysical generation and control of music. It makes use of muscle sounds (MMG^[1]) produced by a performer as both raw sonic material and control data. Presently the Xth Sense (XS) technology consists of low-cost, wearable biosensors and a Pure Data based application for capture, analysis, real time processing and playback of human muscle sounds. The technical implementation of the XS biosensors has been recently illustrated^[2].

This paper describes the design of the XS software; it is a program that enables a computer to “listen” to the MMG signals transduced by the XS biosensors, to understand the performance main features, and therefore to interact with the performer. After a brief introduction on the nature of the interaction fostered by the XS technology, I focus on the framework main features such as: the XS library, a tabbed dynamic interface (TDI), a MMG features extraction unit, and a graph-on-parent (GOP) routing system for dynamic mapping of gesture to sound.

Music for Flesh II, is a solo sound piece for the Xth Sense.

References

↑ wikipedia:Mechanomyogram
↑ M. Donnarumma: “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” Proceedings of the Linux Audio Conference (LAC'11), 2011

Sponsors and partners of the 4th internationals Pure Data Convention in Weimar 2011

4^th international Pure Data Convention 2011 Weimar ~ Berlin

[1] wikipedia:Mechanomyogram

[2] M. Donnarumma: “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” Proceedings of the Linux Audio Conference (LAC'11), 2011

[1]

[2]

@@ Line 1: / Line 1: @@
-Biosensing musical technologies use biological signals of a human subject to control music. One of the earliest applications can be identified in Alvin Lucier's Music for Solo Performer (1965). Alpha waves generated when the performer enters a peculiar mind state are transduced into electrical signals used to vibrate percussion instruments. Over the past thirty years biosensing technologies have been comprehensively studied and presently notable biophysical-only music performances1 are being implemented at SARC by a research group lead by the main contributor to the Bio Muse project Ben Knapp.
+== A Pd framework for the Xth Sense: enabling computers to sense human kinetic behaviour==
-Whereas biological motion and movement and music are arising topics of interest in neuroscience research, the biologic body is being studied by music researchers as a mean to control virtual instruments. Although such approach has informed gestural control of music, I argue that it overlooks the expressive capabilities of biological sounds produced by the body. They are inaudible but may retain a meaningful vocabulary of intimate interactions with the musicians' actions.
+Author: Marco Donnarumma
-To what extent could biologic sounds be employed musically? In which ways could the performer's perceptual experience be affected? How could such experimental paradigm motivate an original perspective on musical performance?
+Download full paper: [[Media:A Pd framework for the Xth Sense-enabling computers to sense human kinetic behaviour.pdf]]
-This paper seeks to outline methods underlying the development of the Xth Sense project, an ongoing research which investigates exploratory applications of biophysical sound design for musical performance and responsive milieux. Presently the Xth Sense technology consists of a low-cost, biosensing wearable hardware device and a Pure Data based application for capture, analysis and real time processing of human muscle sounds.
+The [http://res.marcodonnarumma.com/ Xth Sense] is an interactive system for the biophysical generation and control of music. It makes use of muscle sounds (MMG<ref>[[wikipedia:Mechanomyogram]]</ref>) produced by a performer as both raw sonic material and control data. Presently the Xth Sense (XS) technology consists of low-cost, wearable biosensors and a [[Pure Data]] based application for capture, analysis, real time processing and playback of human muscle sounds. The technical implementation of the XS biosensors has been recently illustrated<ref>M. Donnarumma: “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” Proceedings of the Linux Audio Conference (LAC'11), 2011</ref>.
-Firstly, an exploratory study of the acoustics of body sounds, namely muscle sounds is illustrated. I describe the development of an audio synthesis model for muscle sounds which offered a deeper understanding of the body sound matter and provided the ground for further experimentations in signal processing and composition in Pd. Then follows a description of the development and design of the Xth Sense - a wearable hardware sensor device for capturing biological body sounds. Eventually I describe the development of the DSP modular application, focusing on its main features such as: a MMG features extraction methods, a GOP routing system for dynamic mapping of gesture to sound, real time audio processing models and GUI utilities, which were developed to provide a fast prototyping environment.
+This paper describes the design of the XS software; it is a program that enables a computer to “listen” to the MMG signals transduced by the XS biosensors, to understand the performance main features, and therefore to interact with the performer. After a brief introduction on the nature of the interaction fostered by the XS technology, I focus on the framework main features such as: the XS library, a tabbed dynamic interface (TDI), a MMG features extraction unit, and a graph-on-parent (GOP) routing system for dynamic mapping of gesture to sound.
-In conclusion, I elaborate on the implementation of the Xth Sense framework in the composition of Music for Flesh II, a solo musical performance; which realization process prompted a better understanding of muscle sounds capabilities for biophysical generation and control of music, thus providing further prospects for the inquiry.
+[[PDCON:Concerts/Marco Donnarumma|Music for Flesh II]], is a solo sound piece for the Xth Sense.
+<videoflash type="vimeo">36962709|700|400</videoflash>
+== References ==
+<references/>
+{{Template:PdCon11}}