PDCON:Conference/Santiago - Making music with biological neural networks in Pd-Gem

SANTIAGO – making music with neural networks in Pd-Gem

Author: Hernán Kerlleñevich

Note: Only this extended abstract is available.

In this paper we present SANTIAGO, a biological neural network environment built in Pd-Gem. The interface focuses in the vast potential for interactive sound art creation emerging from biological neural networks, as a paradigmatic complex system for musical exploration. Our motivation relies upon the idea of relating metaphorically neural behaviors to electronic and acoustic instruments notes, by means of flexible mapping strategies. In this spirit, we introduce a simple, yet dynamically rich, neural system to develop new interfaces for generative music composition and performance.

The interface is named SANTIAGO, after the renowned Spanish physiologist Santiago Ramon y Cajal. It consists of a modular patch, including a core for biological neural network simulations and diverse input/output modules that can be mapped to the desired musical parameters, as pitch, duration, intensity, timbre, beat, etc. The network consists of units (neurons) connected by unidirectional links (synapses) and are characterized by a continuous level (voltage). The neural model used is a two dimentional simple one, but it still mimics the activity of real biological neurons.

Every neural spike in SANTIAGO produces a rhythmic event, in contrast with regular artificial neural networks. These rhythmic streams can vary from periodic or quasi-periodic to chaotic or stochastic and also generate polirhythmicity. This behavior results neither too random nor too uniform and can be modiﬁed in an interactive way as the network evolves. The activity of the network, for instance, could be tuned to be minimal or dense generating different sound landscapes.

The patch has a GUI-workspace that allows the control of every parameter in the neuron models, the network properties and the sonification outputs. The user can intuitively design simple to complex network configurations by dynamically creating neurons and configuring their inter-connectivity. The workspace has a preset manager for every module and allows the user to store presets and interpolate between different network or neural settings as it sounds. The instrument section provides panels for several sound parameters and a mixer panel serves as the final stage for signal routing, processing and spatialization in ambisonics format. The system also provides visualization tools, thus allowing not only a better understanding of the network activity through visual feedback, but also highly customizable audiovisual performances.

Multiple instances of SANTIAGO can be loaded in different machines, thus allowing distributed computing. Having multiple signal and data inputs and outputs, as well as standard communications protocols such as MIDI, OSC and TCP/IP, it becomes a unique tool for composers and performers, suitable for different performance scenarios, like live electronics, sound installations and telematic concerts.