GMU:Tutorials/Performance Platform/Recognizing Gestures with Wekinator

From Medien Wiki
< GMU:Tutorials
Revision as of 14:22, 20 July 2016 by Rachel (talk | contribs) (Created page with "First Download Wekinator and openFrameworks (you will need to also download an IDE such as Xcode in order to run openFrameworks) Instructions for downloading and setting up both...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

First Download Wekinator and openFrameworks (you will need to also download an IDE such as Xcode in order to run openFrameworks)

Instructions for downloading and setting up both of those can be found here: http://www.wekinator.org/downloads/ http://openframeworks.cc/download/

For Facial Gestures, you can use FaceOSC which runs on openFrameworks and can be downloaded here https://github.com/kylemcdonald/ofxFaceTracker/releases

This programme can be used with a webcam and tracks the position data of facial features, sending the values as OSC.

Tutorialpic1.png

Now, open Wekinator and listen for incoming OSC messages from FaceOSC. It will send 66 pairs of xy values, i.e 132 values each frame. Set Wekinator to listen to port 8338 and for the input message /raw, and choose 132 for number of inputs. For this example I have chosen 1 output with 3 classes under a classifier output. (You can choose anything depending on what sort of output you want). This output will assign one of three images to three different facial expressions.

Tutorialpic2.png

In Wekinator, click next. You will now come to a screen with a dropdown menu next to outputs. Select 1, make your desired facial expression and press start recording. Try recording around 20 examples and then select 2 in the dropdown menu. Now make your second desired facial expression and record again. Do this again for option 3. Now press Train. When training is finished, the status circle will turn green and you can press run. You should notice that when you make one of your three chosen facial expressions, the dropdown menu automatically changes to 1, 2 or 3. If this is happening for the wrong expressions, you may need to add more examples. If this is the case, stop running and repeat the previous steps. Keep adding examples until you get the accuracy you want.

Tutorialpic3.png

Now for an output! Here is a simple output example I made in processing for one classifier with three classes. Just load the sketch with three images and change “happy.jpg" etc to match your file names.

Tutorialpic5.png Tutorialpic4.png

IMPORTANT: Wekinator sends values as floats, not ints - so if you are writing an output sketch, you will have to convert.

If you have, for example, chosen three pictures of your dog, you could achieve something like this.

Tutorialpic6.png Tutorialpic7.png Tutorialpic8.png