View source for GMU:Tutorials/Networking/Controlling Skeletons in Unity with The Captury

==Introduction==
This tutorial will show you how to set up a rigged Model in Blender, how to export it into Unity and how to control it with The Cpatury's OSC Messages.
<br><br>

==Shortcuts (Blender)==
'''TAB''' = Toggling between Edit- and Object-Mode<br>
'''Shift + Right-Click''' = Multi-Select<br>
'''Spacebar''' = Open Finder -> type in commands and select them<br>
'''X''' = delete<br>
'''S''' = scale<br>
'''Shift + D''' = Duplicate Object<br>
'''Ctrl + J''' = Combine Objects to one Mesh<br>
'''G''' = "Grab"<br>
<br><br>

==Creating the Rig in Blender==
<br>
'''Step1'''
<br>
Open a new .blend File. Delete the Cube which is set as default with "x". 
Go to Add -> Armature and select Human (MetaRig).<br>
If there is no Rig then go to: File -> User Prefrences and select the Addons Tab and type in "Rigify". Make sure the Addon is checked.<br>
If not simply click on the square and Save User Settings at the bottom.
[[File:create_rig.png|thumb|320px|left|Create the Rig]]
[[File:check_rigify.png|thumb|320px||left|Go to User Prefrences]]
[[File:check_rigify_activated.png|thumb|300px||left|Check if Rigify is activated]]
[[File:basic_rig.png|thumb|320px||left|This is how the Rig looks]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step2'''
<br>
Select your Rig and switch to the Outliner on the right. Double click on metarig and type in the name you wish: e.g "Cubeboy".
[[File:rename_your_rig.png|thumb|320px||left|Rename your Rig]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step3'''
<br>
Now switch back to the 3d-Rig and select it with right-click and press "TAB" to switch into "Edit Mode".<br>
Because we don't need any fingers and unnacessary bones we are going to delete the Fingers: Hold "Shift" and right-click to multi-select the bones.<br>
After you selected all fingers go and press "x" to delete the bones. Repeat the Process on the other side. We are also going to delete the "Heel bones".
[[File:multiselect.png|thumb|320px||left|Multiselect "Shift + Right Click"]]
[[File:delete_bones.png|thumb|320px||left|Delete bones with "X"]]
[[File:delete_heels.png|thumb|320px||left|Deleting the Heels]]
<br>
<br>
<br style="clear:both">
<br>
<br>
Our Rig should now look like this (I expanded the bone structure in the "outliner" by clicking on the plus sypmbol).
[[File:bone_structure.png|thumb|320px||left|Bone Structure]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Quick Helper'''
<br>
Sometimes you encounter that this little red and white rounded thing was moved accidently from it's original position which is actually the "Cursor"<br>
centerd in the X,Y,Z Coordinate = 0,0,0. If you press Spacebar you can simply type in "Snap Cursor to Center" and click on the slected Field to reposition it.<br>
This is Importanrt because when we create a new Object we want it to be fully centerd. (This will make life easier).
[[File:readjust_cursor.png|thumb|320px||left|Set cursor to 0,0,0]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step4'''
<br>
Switch back to "Object Mode". Go to the Properties Window on the right under the Outliner and click the "Armature" tab. 
Activate X-ray so you can watch trough the Mesh afterwards.
[[File:activate_xray.png|thumb|320px||left|Activate X-Ray]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step5'''
<br>
We are going to add now the Mesh for our Cubeboy so it will be visible in "Unity" afterwards because we won't see the bones.<br> 
So go again to "Add" in your 3D-View and add a Cube. Be sure to be in "Object Mode".<br>
Scale it down till it fits as a had with "S". Move it up to the "Head-Bone" with left clicking the blue "Z-Axis" Arrow.
[[File:add_cube.png|thumb|320px||left|Add Cube in Object Mode]]
[[File:positioning_cube.png|thumb|320px||left|Position and Scaling of Cubes ]]
<br>
<br>
<br style="clear:both">
<br>
<br>
Our next step would be to duplicate the main Cube and put it on some parts of the Rig. Press "Shift + D" to duplicate the cube.<br>
The new Cube will be outlined with white. Now press "Z", "Y" or "X" to move it along one Axis and confirm it with "Left-Click".<br>
Repeat these steps till you are happe with your Cubeboy.
[[File:duplicate_object.png|thumb|320px||left|Duplicate Cubes]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step6'''
<br>
Mr. Cubeboy should now look like this. What we need now is to combine all the cubes because we only need one Mesh.
<br>So select all the Cubes and press "Ctrl + J"
to combine them. Go to the Outliner and Rename the "Cube.015"  to e.g "Cubeboy_Mesh".
[[File:cubeboy.png|thumb|320px||left|How Cubeboy should look like]]
[[File:combine_mesh.png|thumb|320px||left|Combine all the Cubes into one Mesh]]
<br>
<br>
<br style="clear:both">
<br>
<br>
'''Step7'''
<br>
In order to move our Rig together wit the Mesh we have to combine both.
First select the Mesh and at last the Rig and combine them with "Ctrl + P" and select "With Automatic Weights".<br>
Blender will set the weights of each bone so you don't have to do it manually.<br><br>
To be sure everything went well switch to "Pose Mode". When we now select a bone it will be rendered blue.<br>
With "G" you can grab and move around the bones with your Mouse. Release them with Right-Click so they get back in Position.
[[File:pose_mode.png|thumb|320px||left|Switch to Pose Mode]]
[[File:select_bone_pose_mode.png|thumb|320px||left|bone Selection Pose Mode]]
[[File:check_animated_bones.png|thumb|320px||left|Check if Bones are animation ready]]
<br>
<br>
<br style="clear:both">
<br>
<br>

==Exporting the Rig==
<br>
After everything is checked go back into Object-Mode and select the Rig and the Mesh. Go to File -> Export -> .fbx <br>
In Export Settings we only need the Armature and Mesh. Notice also make sure that "Selected Objects" is checked. Name your .fbx file properly and export it.
[[File:file_menu_export.png|thumb|320px||left|How to Export]]
[[File:export_settings.png|thumb|320px||left|Export Settings]]
<br>
<br>
<br style="clear:both">
<br>
<br>
==Importing the Rig in Unity==
<br>
Open Unity and create a new project. Call it as you like.
[[File:create_new_project.png|thumb|320px||left|Creat New Project]]
<br>
<br>
<br style="clear:both">
<br>
<br>
Right click in your Assets folder -> Create -> Folder and name it. "Model".<br> 
To import your Model simply drag it from your destination Folder into Model or right click "Import new Asset" and select your fbx file.<br>
Drag and Drop the Model into the "Hierarchy" on the left side. Move the character towards the Camera so we can see it better.
[[File:create_folder.png|thumb|320px||left|Prepare Folder Steructures]][[File:import_model.png|thumb|320px||left|Import_Model]]
<br>
<br>
<br style="clear:both">
<br>
<br>
Once we extend the Cubeboy's Rig in the Hierarchy we can see our Bones. 
Ignone the once with a "_end" Tag. You also notice why we called our Mesh "Cubeboy_Mesh" so we can't get confused.
[[File:cubeboy_hierarchy.png|thumb|320px||left|Hierarchy of Bones]]
<br>
<br>
<br style="clear:both">
<br>
<br>
==Setting up the Scripts==
<br>
Our next step will be to prepare an empty Gameobject where we will put in our OSC Scripts. <br>Go to Hierarchy right click -> "Create Empty" and call it "OSC_Script".
[[File:create_empty.png|thumb|320px||left|Create an Empty Game Object]]
<br>
<br>
<br style="clear:both">
<br>
<br>
Download the OSChandler Files on Github [https://github.com/heaversm/unity-osc-receiver | OSC for Unity] and Copypaste the Plugin Folder from Assets into your Asset Folder.<br>
Download this modified [[Media:OSC_Reciever.zip | OSC_Reciever Script]] and paste it to a new Folder called "Scripts".
Select your Empty called "OSC_Script" and click on "Add Component" and search for all three Scripts and add them. It should look like this:
[[File:add_scripts.png|thumb|320px||left|Adding Scripts to Object]]
[[File:osc_setup.png|thumb|320px||left|Script Setup]]
<br>
<br>
<br style="clear:both">
<br>
<br>
==Bone Overview The Captury==
<br>
Switch to The Captury and go into Playback Mode. Select your Recording.<br>
On the right Panel you can see the "Skeleton Types" tab. Click on it and a whole list will appear which every single bone.<br> 
The bone names we need are the ones with the white background. These are the bones which we need to set up in our OSC_Reciever script.
[[File:playback_mode.png|thumb|400px||left|Playback Mode]]
[[File:skeleton_types.png|thumb|400px||left|Skeleton Types]]
<br>
<br>
<br style="clear:both">
<br>
<br>
==Modifying the OSC Script in Unity==
<br>
Since these steps are going to be repetitive I will show you how to set up two bones for the OSC Messages between Unity and The Captury.<br>
Open the OSC_Reciever script in Unity by default with "Viusal Studio" or "Monodevelop". Go to line "60" and "63" and change the "yourname" according to your recording.
<source lang="javascript">
    msg = handler.StringToOscMessage("/subscribe/yourname/blender/Root/vector 50.0 0.0 20.0"); 
    handler.Send(msg);

    msg = handler.StringToOscMessage("/subscribe/yourname/blender/Head/vector 50.0 0.0 20.0");
    handler.Send(msg);
</source>
Also change it in line "101" adn "107".
<source lang="javascript">
    switch (msgAddress){

     
        case "/yourname/blender/Root/vector":
            hips_x = values[0];
            hips_y = values[1];
            hips_z = values[2];
            break;

        case "/yourname/blender/Head/vector":
            head_x = values[0];
            head_y = values[1];
            head_z = values[2];
            break;

        default:
            //
            break;
    }
</source>
<br>
'''Notice''': You can also change the Bone types coming from The Captury. E.g you don't want to animate the "Head" bone, change it to which ever bone you like. Save your Script.<br><br>
Now go back to Unity. In the Editor on the right. Once the OSC_Script is selected, type in the bones from your Model not the ones from The Captury.
You can see them in the Hierarchy.
[[File:osc_setup.png|thumb|320px||left|Modifying the Script]]
<br>
<br>
<br style="clear:both">
<br>
<br>
==Run Everything==
<br>
Be sure everything is set up correctly (Message string / Bonenames / Modelbones / Ports / Remote IP) before you run The Captury and Unity.
<br>
Run The Captury in "Playback Mode" looping. Run Unity. Your Character shall move now.
<br>
<br>
<br>
'''Thank you!'''