Ben & Lucas OSC Game Documentation

"Give It To Me Baby"

We did a little game, where you can control a dancefloor in a Unity Scene, with microphone input. The player is requested to shout, beatbox or clap in front of the microphone, to move the dancers on the screen.

An audio signal is grabbed in Max/Msp and then sent to Unity via OSC. In Unity the signal is used to control an object, which serves as the dance-floor for some SpriteAnimation of Dancers. When the floor moves the dancers get kicked around.

When you start the game, there is music but no beat. You have the control over the beat. When you start using the mic, the beat is un-muted. When you stop interacting, the message - "Give It To Me Baby!" - appears, which forces you to go on.

- -

Max/Msp

Grabbing the Audio Input

the Master Patch

In Max you can grab any audio input. In our case we use a microphone connected to an audio interface.
The audio signal is then converted into a float value stream. With a threshold patch, we can set the minimum of input to trigger a bang. The bang from the threshold we then send to a message box, containing a predefined OSC message string ( /goOnBang $1 ).

When a peak is detected, the Max patch also sends a message ( /Ben/x $1 ) to Unity via OSC. "$1" is a variable which is replaced by a float "1.3". This value is defines the amount of height, and therefore how high the dancefloor gets lifted.

Setting the threshold

setting the threshold

To use a constant audiostream, and trigger things with it, you need something called "Envelope Follower". Therefore we used a sub-patch which comes with to standard max-externals. This patch detects peaks in the audio-signal, if you want to use a peak above a certain level you have to look at the gain and set the threshold then.

Trigger Text and Mute Audio Sources in Unity

sending a bang, a pause signal

We use the detected peaks in the mic signal, to trigger messages and mute the beat-loop in Unity. Therefore we send the OSC message ( /goOnBang $1 ) to the PC running Unity, via OSC. When the player pauses for 350ms, the beat-loop stops, and the message - "Give It To Me Baby" - appears on the screen. When another peak is detected, the Max patch sends a signal to Unity, which tells it to go on, with playing the beat-loop and the message on the screen disappears.

---

Unity

Creating Sprite Animations

Creating the Spritesheet

In order to create a sprite animation we used an open source tool called "Aseprite". It's like Photoshop but more intuitive when it comes to pixel art. Its easy to work with and also provides a good way for frame by frame animation. IMPORTANT: Export the file as a PNG before you Import it into unity

Preparing the Sprites in Unity

Creating the Spritesheet
Creating the Spritesheet

Once you imported a png