435
edits
Pedroramos (talk | contribs) |
Pedroramos (talk | contribs) |
||
(14 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
= | =Machine Learning and Max/MSP integration for the development of interactive audiovisual systems= | ||
==Objective== | |||
The project consisted on a research regarding the connection and possible applications of Machine Learning in Max/MSP/Jitter, as well as the development of an environment inside of Max/MSP that could demonstrate and prototype such application. The final stage of the development consisted on getting sensor signal, in the case an Arduino, to receive physiological and potential environmental signal, and conduct it through a stochastic sound generation method in Max/MSP which the reactions were determined by Machine Learning. | The project consisted on a research regarding the connection and possible applications of Machine Learning in Max/MSP/Jitter, as well as the development of an environment inside of Max/MSP that could demonstrate and prototype such application. The final stage of the development consisted on getting sensor signal, in the case an Arduino, to receive physiological and potential environmental signal, and conduct it through a stochastic sound generation method in Max/MSP which the reactions were determined by Machine Learning. | ||
Line 30: | Line 31: | ||
The brief explanation on some of the aspects just explained, for the work of Wekinator, in which the training is itself limited regarding the specifications on some of such aspects, such as weights, number of layers on Neural Network training, amongst others, can however be important. Although the software works as a black-box, that means, it’s not really possible to grasp everything that is happening on an algorithmically level, and for such gives the creator less autonomy on the training process and possible outputs, it although already offers the possibility of working with different types of Training Methods, such as Neural Networks and Linear or Polynomial Regression. For such, and in order to achieve bigger autonomy on the training processes despite such limitations, the comprehension of some key terms on Machine Learning can make itself necessary or at least helpful, namely: classifiers, Backpropagation, decision stumps, amongst others. | The brief explanation on some of the aspects just explained, for the work of Wekinator, in which the training is itself limited regarding the specifications on some of such aspects, such as weights, number of layers on Neural Network training, amongst others, can however be important. Although the software works as a black-box, that means, it’s not really possible to grasp everything that is happening on an algorithmically level, and for such gives the creator less autonomy on the training process and possible outputs, it although already offers the possibility of working with different types of Training Methods, such as Neural Networks and Linear or Polynomial Regression. For such, and in order to achieve bigger autonomy on the training processes despite such limitations, the comprehension of some key terms on Machine Learning can make itself necessary or at least helpful, namely: classifiers, Backpropagation, decision stumps, amongst others. | ||
=== | ===Training with Wekinator=== | ||
Considering the short incursion on generals on Machine Learning, and the decision on working with Wekinator considering the opportunities it offers, and its limitations, a short description on the connection between Max/MSP and Wekinator might be made necessary. Considering the last developed patch for the system, follows a brief step-by-step on the Inputs, Model Processing and training, and Output back to Max/MSP. | Considering the short incursion on generals on Machine Learning, and the decision on working with Wekinator considering the opportunities it offers, and its limitations, a short description on the connection between Max/MSP and Wekinator might be made necessary. Considering the last developed patch for the system, follows a brief step-by-step on the Inputs, Model Processing and training, and Output back to Max/MSP. | ||
Line 69: | Line 70: | ||
[[File:screen_arduino.mov|700px]] | [[File:screen_arduino.mov|700px]] | ||
===Training | ===Machine Learning Training with Max/MSP and Wekinator=== | ||
A demonstration on the training process and the subsequent parameters control on Max/MSP after the training process. | |||
One of the main objectives of the process was to set a successful cycle of signal processing flowing between the involved elements, and developing a functioning prototype consisted on demonstrating the possibilities of application of such system for a variety of other possible applications developed within or with Max/MSP. | |||
A recorded short clip also demonstrate some of the audio outputs from the training and subsequent parameter manipulation after the training, that consisted assigning different randomised output values to each oscillator in accordance to varying values of the gain sliders. | |||
[[File:training2.mp4|600px]] | |||
[[File:training_audio_2.wav]] | |||
==Max/MSP Patches== | ==Max/MSP Patches== | ||
Line 97: | Line 108: | ||
[[Media:Arduino-sensors-max.ino|Max-Arduino Sensor Connection]] | [[Media:Arduino-sensors-max.ino|Max-Arduino Sensor Connection]] | ||
==References | ==References and Bibliographic References== | ||
Camastra, Francesco; Vinciarelli, Alessandro. Machine Learning for Audio, Image and Video Analysis. Theory and Applications, Second Edition, 2015. | Camastra, Francesco; Vinciarelli, Alessandro. Machine Learning for Audio, Image and Video Analysis. Theory and Applications, Second Edition, 2015. | ||
* Guy Ben-Ary [http://guybenary.com/work/cellf/#Neural_Network_–_Analogue_Synth_Interface|* CellF Neural Synth] | * Guy Ben-Ary [http://guybenary.com/work/cellf/#Neural_Network_–_Analogue_Synth_Interface|* CellF Neural Synth] |
edits