The Verdant Symphony

This project fuses nature, technology, and music into a unique blend. Six plants, equipped with gas sensors, guide motors that pluck guitar strings, each producing an individual tone. The result is a living symphony, where the natural signals of plants shape a new fusion of nature, sound and the digital world.

Brainstorming and concept development

Inspired by some exemplary works presented during the first two seminar sessions, Hanna quickly came up with the idea of having plants play the guitar together. I was immediately fascinated by this idea and very grateful that Hanna wanted to collaborate with me. Together, we developed the concept further week by week. I documented and organized this thought process and the resulting ideas on a Miro board:

Miro Board

Initially, the plan was for two plants to interact and control the guitar. Later, we considered having a guitar and a computer-generated voice play music together, but we eventually abandoned this idea. In the end, we settled on the concept of six plants, each playing a single guitar string, controlled by the gas emissions of the respective plant. This idea was then presented during the midterm presentation. Hanna had created an initial version of the code, while I had already started working on the structural implementation of a mount for the motors and had designed a construction plan for it.

(written by Fiene)

Technical implementation

Embedded systems development

During our project, we used Visual Studio Code and Arduino to program and control motors based on sensor data. Our first challenge was to determine how the motor should move. We aimed to ensure that the motors moved only once per activation, producing a single tone. To achieve this, I created an initial test code to establish how to instruct the motor to move right when it was on the left and vice versa.

Next, we needed to develop a pattern to simulate incoming sensor data. The second test code introduced random numbers to help us simulate a trigger value for the motors. This was combined with our previous motor movement code to create a preliminary response system.

I then worked on enabling two motors to move independently. To accomplish this, I implemented two different random numbers so that each motor would move at a different "trigger value". Adjustments were made to delay times to introduce a more natural, randomized effect.

To acquire real sensor data, we opted for an MQ-2 gas sensor, which detects combustible gases. We initially attempted to retrieve analog output values from one sensor. Since the raw sensor data wasn’t as representative as we needed, I decided to convert the readings into ppm (parts per million) for more meaningful analysis.

The next step was to replace our previously used random numbers with actual gas concentration values measured by the sensor. I then expanded the system to incorporate six sensors and six motors, which required multiple iterations to perfect. A major challenge was implementing all sensor values, motors, and their corresponding variables into a function that allowed each sensor-motor pair to operate independently. We replaced the general delay time in the loop with millis() to enhance achieve that.

After measuring the gas concentrations produced by plants over time, I made several adjustments to optimize the installation. Each motor was assigned an individual trigger value based on the gas readings from its respective sensor. Since sensor readings were taken at short intervals and did not fluctuate significantly, motors would have been triggered too frequently, leading to excessive noise. To mitigate this, I introduced individual delay times for each motor before they could be triggered again. To create a more natural and less predictable sound pattern, I introduced a "random factor". The motor's delay time was then multiplied by this factor which was recalculated with each sensor reading, ensuring a more varied and pleasant auditory experience.

Debugging print commands were added to the code to help monitor sensor functionality when connected to a computer. These were primarily for troubleshooting and not necessary for the final public installation. Although the code could have been optimized for brevity, I chose to keep it more detailed to maintain a clear overview of all sensors, motors, and timing adjustments. In the end, our system successfully produced tones in a random frequency, creating a dynamic and immersive experience.

Technology

ESP32
MQ-2 sensor for combustible gas
SG90 9g Micro Servo

Code

For the platformio.ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps = roboticsbrno/ServoESP32@1.0.3
monitor_speed = 9600

Fort he main.cpp:

#include <Arduino.h>
// include the servo library
#include <Servo.h>

//  timing 
unsigned long currentTime = 0;

unsigned long lastTimeA = 0;
unsigned long lastTimeB = 0;
unsigned long lastTimeC = 0;
unsigned long lastTimeD = 0;
unsigned long lastTimeE = 0;
unsigned long lastTimeF = 0;

const int baseServoDelayTimeA = 17300;
const int baseServoDelayTimeB = 11020;
const int baseServoDelayTimeC = 15400;
const int baseServoDelayTimeD = 12800;
const int baseServoDelayTimeE = 8400;
const int baseServoDelayTimeF = 6300;

float randMultiplier = 1.0;

long randNumber; 

//sensors 
const float sensorTriggerValueA = 700;
const float sensorTriggerValueB = 520;
const float sensorTriggerValueC = 650;
const float sensorTriggerValueD = 880;
const float sensorTriggerValueE = 950;
const float sensorTriggerValueF = 1100;

float getSensorValuePpm (int sensorPin);

const int sensorA = 36;
const int sensorB = 39;
const int sensorC = 34;
const int sensorD = 35;
const int sensorE = 32;
const int sensorF = 33;

float gasValueSensorA = getSensorValuePpm (sensorA);
float gasValueSensorB = getSensorValuePpm (sensorB);
float gasValueSensorC = getSensorValuePpm (sensorC);
float gasValueSensorD = getSensorValuePpm (sensorD);
float gasValueSensorE = getSensorValuePpm (sensorE);
float gasValueSensorF = getSensorValuePpm (sensorF);

//servos 
const int servoPinA = 19;
const int servoPinB = 18;
const int servoPinC = 05;
const int servoPinD = 17;
const int servoPinE = 16;
const int servoPinF = 04;

Servo servoA;
Servo servoB;
Servo servoC;
Servo servoD;
Servo servoE;
Servo servoF;

boolean isLeftA;
boolean isLeftB;
boolean isLeftC;
boolean isLeftD;
boolean isLeftE;
boolean isLeftF;

//function declarations 
void playSound(unsigned long &lastTime, int sensorPin, Servo &servo, float &gasValueSensor, boolean &isLeft, int baseServoDelayTime, float sensorTriggerValue);

float getSensorValuePpm (int sensorPin);

void setup() {
  analogSetAttenuation(ADC_11db);
  Serial.begin(9600);

  servoA.attach(servoPinA);
  servoA.write(30);
  isLeftA = true;

  servoB.attach(servoPinB);
  servoB.write(30);
  isLeftB = true;

  servoC.attach(servoPinC);
  servoC.write(30);
  isLeftC = true;

  servoD.attach(servoPinD);
  servoD.write(30);
  isLeftD = true;

  servoE.attach(servoPinE);
  servoE.write(30);
  isLeftE = true;

  servoF.attach(servoPinF);
  servoF.write(30);
  isLeftF = true;
}

void loop() {
  currentTime = millis();

  playSound(lastTimeA, sensorA, servoA, gasValueSensorA, isLeftA, baseServoDelayTimeA, sensorTriggerValueA);
  playSound(lastTimeB, sensorB, servoB, gasValueSensorB, isLeftB, baseServoDelayTimeB, sensorTriggerValueB);
  playSound(lastTimeC, sensorC, servoC, gasValueSensorC, isLeftC, baseServoDelayTimeC, sensorTriggerValueC);
  playSound(lastTimeD, sensorD, servoD, gasValueSensorD, isLeftD, baseServoDelayTimeD, sensorTriggerValueD);
  playSound(lastTimeE, sensorE, servoE, gasValueSensorE, isLeftE, baseServoDelayTimeE, sensorTriggerValueE);
  playSound(lastTimeF, sensorF, servoF, gasValueSensorF, isLeftF, baseServoDelayTimeF, sensorTriggerValueF);
}

//functions

void playSound(unsigned long &lastTime, int sensorPin, Servo &servo, float &gasValueSensor, boolean &isLeft, int baseServoDelayTime, float sensorTriggerValue){
  
  //generate multiplier and adjust delaytime 
  randMultiplier = 0.7 + (random(0, 341) / 100.0);
  int adjustedDelayTime = baseServoDelayTime * randMultiplier;

  if (currentTime - lastTime >= adjustedDelayTime){
      // check if timing works
      Serial.print("Random delayTime");
      Serial.println(adjustedDelayTime);
      Serial.println("Triggering sensor reaading");
      gasValueSensor = getSensorValuePpm(sensorPin);
      // check sensor value 
      Serial.print("Sensor Value: ");
      Serial.println(gasValueSensor);

        if (gasValueSensor > sensorTriggerValue){
          Serial.println("Triggering servo");
          // move servo to play sound
            if (isLeft == true){
              servo.write(0);
              isLeft = false;
              Serial.println("right");
            }
            else{
              servo.write(30);
              isLeft = true; 
              Serial.println("left");
            }
            Serial.println("Servo moved. Last Time updated.");
          }
          // update lastTime
          lastTime = currentTime;
        }
}

float getSensorValuePpm (int sensorPin) {
  float voltage = analogRead(sensorPin) * (5.0 / 1023.0);
  float RS_air = 10.0;
  float RL = 2.0;
  float ratio = RL / RS_air;
  float RS = ((5.0 / voltage) - 1.0) * RL;
  float ppm = 1000.0 * pow((RS / RS_air), ratio);
 return ppm;
}

(written by Hanna)

Manual construction

I was responsible for the structural and mechanical implementation of the project since my strengths lie far more in this area than in writing computer code. Additionally, unlike Hanna, I have the necessary resources at home to craft.

We needed a mount that would allow the servo motors to hover above the guitar strings so they could pluck them individually. The servo motors had to be positioned at a certain distance from each other to avoid interfering with one another when striking the strings. We had already identified this issue through a cardboard prototype. Since guitar picks were to be attached to the motors, increasing the radius of the strike, it made sense to always leave two strings between those being played.

I decided to attach two motors to one bar. The construction sketch can be seen here:

Since I couldn’t guarantee millimeter-precise construction, and factors like the surface, variations in the way the picks were attached, or different motor models could create height differences, it was necessary to allow for fine adjustments to the height of the mount. For this, I used special adjustment screws that allow the height of the mounting beams to be individually adjusted on both sides. Unlike regular screws, these have two types of threads: the upper thread remains fixed in the wood, while the lower thread can be screwed in and out like a standard screw. This enables precise millimeter-level height adjustments to fit the conditions.

Additionally, the sensors had to be connected to the breadboard. To do this, I soldered cables—each consisting of three wires—of sufficient length to each of the six sensors. I then soldered a connector to each wire to facilitate easy attachment to the breadboard.

(written by Fiene)

The Plants

Hyacinth
Euphorbia
Fern
Begonia
Peperomia
Alocasia

Exhibition

Concept

The idea was to place the guitar on a table, with a clothing rack positioned behind it, from which the six plants would hang. These included a hyacinth and an alocasia in bags, a fern and a pepper plant in small greenhouses, as well as a begonia and a moldy euphorbia in screw-top jars.

The cables and the breadboard were intentionally installed with minimal cable management, keeping them clearly visible to emphasize the DIY aesthetic.

The lighting needed to be bright enough to provide sufficient light for the plants without being too harsh. Hanna brought her grow lights, which perfectly illuminated the exhibition niche where our work was displayed.

We chose the niche as our exhibition space because it allowed us to maintain a necessary distance from other sound-based projects. Additionally, this setup ensured that the work could only be viewed from the front, preventing visitors from accidentally bumping into it and disturbing any components.

Preparations and setup

Hanna and I met in Weimar on the Saturday before the exhibition to attach the picks to the motors and test the interaction of all components. This allowed us to check the overall functionality of both the code and the construction. Fortunately, everything worked as intended, with only a few minor tasks remaining.

Our plan was to hang the plants from a clothing rack. However, since two of the plants were potted in jars, suspending them proved to be a challenge. To solve this, I crafted wooden discs with holes through which we could thread wire, ensuring that the jars could be securely hung. Hanna also made some minor adjustments to our code.

On the Thursday before the exhibition, we finalized our setup. We darkened the window, adjusted the lighting, secured all components to the table to prevent any shifting during the exhibition, connected the sensors and motors, plugged in the power supply, set the timers, and fine-tuned the height of the mounting structure to ensure that each pick could strike its string and produce a pleasant sound.

(written by Fiene)

Winterwerkschau

The Winterwerkschau was a great success for our project. Contrary to our initial concerns, the adhesive held up perfectly, keeping the picks securely attached to the motors for both days of the exhibition. As a result, all the strings could be continuously played without any issues.

When Hanna and I took our turn overseeing the exhibition, we were met with an incredible amount of interest and enthusiasm. Visitors were fascinated by our work, showering us with questions and curiosity. It was an incredibly rewarding experience that filled us with pride and reinforced our excitement about what we had created.

All the hard work had truly paid off, and seeing our project resonate with others made the effort even more worthwhile.

(written by Fiene)