GMU:Sensor Hacklab/Christian Doeller

Project Overview

I am interested in exploring different ways of interaction between humans and their everyday environment of things.

In this case I am focusing on the common activity of walking in connection with the carpet as a customizable floor covering.
I want to create a carpet / a collection of carpets that sense people walking on it / them and then trigger a certain output as a response.

I would like to create a complex environment in which people have to move through the space in order to explore its different features.
Therefore I want to explore possibilities of creating an unpredictable output environment by introducing a certain amount of coincidental behavior.

The minimum goal of this course is creating a single interactive carpet.

This idea is related to another project in which I designed a kind of yoga mat.
I did not yet include electronics in the mat because I am doubtful about the determined amount of switches and their pre defined outputs.
The interactive carpet is somehow an attempt to find out what happens if electronics are included and how the outputs can become unpredictable.

Technical implementation

I want to construct a matrix structure in order to implement a „multiple switch device“ that can be triggered by peoples walking on it.
Therefore I want to design switches that can handle the weight of a person standing on them.
The switches have to be made of a flexible material so that they automatically return to their „off-state“ when the person changes his/her location.

I will try to implement a kind of resistor ladder in order to reduce the amount of input pins needed.
This amount can easily become very high depending on the amount of sensors.
The outputs will be controlled by an Arduino Micro controller board.
It will trigger certain sounds or other outputs such as motor movement or LED lights.

I accomplished different experiments in order to outline the needs of my final idea:

1) exploring touch sensor material
2) creating a simple button matrix / button resistor ladder
3) hacking a toy keyboard + exploring its inner architecture
4) reverse engineering and connecting a keyboard matrix with Arduino + exploring its functions
5) the toy keyboard matrix as a resistive sensor 6) exploring different ways of implementing switches in a carpet

Protoyping experiments

1) DIY Sensor Materials

During the first class we investigated different materials and their conductivity / resistivity.
I was mostly interested in materials that produce variable resistance when they are pushed and squeezed
like „resistive plastic foam“.

Another nice discovery was combining a metallic sponge and a piece of aluminium foil.
The outcome is a sensor that reacts to force.

2) „Button Resistor Ladder“

The first experiment including matrix structures was about a chain of buttons.
Each of these buttons were connected via different resistor values.
The last resistor of that kind of „resistor ladder“ connects the whole structure to the Arduinos analog input
via a voltage divider with a 100K resistor.
—> see the circuit drawing below

This „button - resistor structure“ assures that each button provokes a specific and distinguishable value
in the Arduinos serial reading.
The experiment could lead to a more complex button structure that can be used as an „analog input matrix“.

3) hacking a toy keyboard, triggering it with relays + exploring its inner architecture

In the second experiment we took a closer look at a toy including a keyboard matrix to play a funny collection
of sounds.
At first we traced each tone back to its switch on the matrix.
Then we soldered wires to the sound terminals in order to make the sounds externally accessible.

We had the idea to further influence and control the sounds with an external switching device.
Our solution was a chain of two relays (this one) controlled by the arduino via a darlington transistor ULN2803.
These relays allowed us to turn the sound on and off so that little pieces of the sounds could be cut out.

We stopped our experiment as soon as we realized that we would need a lot of (expensive) relays to control
every available sound on the toy chip.

see "Darlington Transistor" on Wikipedia

After that I decided to explore the toy matrix a little further.
A drawing helped me to understand and trace the connections inside the matrix structure.
I then attached a DIY connector in order to use the bottom connectors.

4) Reverse-engineering and programming a common keyboard matrix

This is a common keyboard matrix with a 4 x 3 buttons number plate.
I „reverse engineered“ the matrix structure (hidden in the case) after I soldered pin headers to its connectors
in order to plug it into the bread board.
I was mostly interested in using the matrix as a variable resistor rather than using it as a „multiple digital
pin input device“. (see experiment no. 2)
I achieved that by connecting different resistor values in between each contact of the vertical and the horizontal
button lines.
In the end a voltage divider helped to create different voltage changes for the Arduinos analog input.
The matrix is actually meant to be used as a "multiple digital pin input / output interface" like shown on this
website (http://pcbheaven.com/wikipages/How_Key_Matrices_Works/) .

In my experiment every button represents a certain value in the analog input.
By doing that I discovered a major problem: some of the values almost overlapped.
Another problem was the amount of noise produced by the analog input.

Therefore I had to define a certain range of values for each button.
Although this worked a bit better there was still a lot of overlap.
This overlap sometimes lead to the effect that the interpretation of buttons being pressed was wrong.

Another interesting part of that experiment was the code.
I did not use any library - instead of that I tried to use my basic programming skills to set it up by myself.
It surprisingly worked out quite well - despite the mentioned overlap problem and the face that I needed
a lot of memory.

In order to solve these issues in the future I need to learn about programming „arrays“ and „forloops“ that
can help minimizing the amount of code.
The overlap problem could be solved by using binary functions / average measurements that define clearer
value states.

see "Arduino Keyboard Matrix Tutorial"

5) the toy keyboard matrix as a resistive sensor

The idea of using a matrix as a variable resistor was a good way of connecting the toy-keyboard to the
arduinos analog input.

My hacked version of the toy keyboard does not really work as a system of clear state switches.
I used a very resistive material in order to access the matrix (graphite lines).
This material in combination with the original matrix circuit material (also very resistive) makes it hard to
define clear input values.

A cool discovery: The multimeter shows that I get a certain value of resistance for each button when all the
vertically and all the horizontally connected „lines“ are connected together.

This way it is possible to use the matrix for instance as a variable resistor for a 555 timer chip in astable
mode between pin 6 and pin 7:

===> VIDEO

6) exploring different ways of implementing switches in a carpet

The final idea after my matrix explorations is to modify our environment by implementing a similar structure
in everyday objects.
I want to use a carpet as the main interface object as it is 1) easy to access and 2) something casual and
omnipresent.
The first problem is to install switches inside / underneath the carpet.
I solved this problem by using an anti-sliding mat that separates the contacts of the switches.
It is a very flexible material that is easy to manipulate without the need of professional tools.
Another problem is designing and building the switches.
The material needs to be 1) flexible (coming back into its original form) and 2) conductive.
I experimented with different materials such as aluminium foil, aluminium plates and a conductive sponge.
My solution consists of a piece of cardboard covered with aluminium foil.

I also discovered that the size of the gaps inside the flexible grid plays a big role in connection to the performance
of the switches.
I tried different sizes until I found a size that responds to the size and pressure of an „average human footstep“.

A larger set-up with 4 switches and matrix-structure:

===> VIDEO

Finishing the Prototype

The plan was to implement more switches into the carpet-matrix in order to allow playing with more output possibilities.

I achieved that by changing the design:

1) Layout
2) Switch Design
3) Programming

1)
I designed a bigger and more complex layout of „switch holes“ inside the „distance grid“.
The switch holes are VERTEILT all over the mat.
I kept all the cut out pieces in order to change the design as I need it by putting them back in place.

2)
I changed the switch design: they now consist of two carpet-size cardboard pieces that are supplied with „circuit lanes“ made of aluminium foil.
There are two cardboard pieces: one is equipped with the bottom part of each switch, the other one is equipped with the upper part.
The bottom part features eight and the upper part features six „circuit lanes“ that are located in a way so that they form a kind of matrix-grid
when they are put on top of each other.

I first planned to use a resistor ladder for the Analog input pin of the arduino in order to minimize the amount of pins being used.

As this did not work out (too many delays in the code, too slow, a lot of noise) I changed the design by attaching a separate
wire to each of the eight „circuit lanes“ that go to the Arduinos digital input.
Now there are 6 + 8 = 14 wires coming out of the carpet. 6 of them are used as „digital out HIGH signals“ and the other 8
are used as „digital in, normally LOW“ pins that check if a button is pressed.

3)
The resistor ladder idea did not really work.
Therefore I decided to get a bit more into working with the Arduino Keypad library.
The library is made for a 4x3 keypad matrix.
This layout can easily be changed by adding more COLS and ROWS to the array.

You can find more information about the library here:

[1]
[2]

External pullup or pulldown resistors do not need to be added.
The code uses the built in IMPUT_PULLUP command.
It is also configured to have a multi button press function for which you normally have to add diodes to each input
in order to prevent short circuits.
I don’t exactly know why but the comments tell that these external diodes do not have to be added when using
the keypad library.