GMU:Tutorials/Networking/Controlling Unity with Processing: Difference between revisions

From Medien Wiki
(→‎Unity: formatting)
 
(3 intermediate revisions by the same user not shown)
Line 20: Line 20:
== Processing ==
== Processing ==
In Processing you need to install a library. Click on ''Tools'' – ''Add Tools'' – Choose ''Libraries'' – Search for oscP5 and install it.<br><br>
In Processing you need to install a library. Click on ''Tools'' – ''Add Tools'' – Choose ''Libraries'' – Search for oscP5 and install it.<br><br>
This is the basic processing script for sending Osc data (it just sets up the connection, actions can be added): <br><code>
This is the basic processing script for sending Osc data (it just sets up the connection, actions can be added): <br>
//load libraries<br>
<source lang="javascript">
import netP5.*;<br>
//load libraries
import oscP5.*;<br>
import netP5.*;
OscP5 oscP5;<br>
import oscP5.*;
NetAddress myRemoteLovation;<br>
 
//make a connection<br>
OscP5 oscP5;
void setup () {<br>
NetAddress myRemoteLovation;
osP5 = new OscP5 (this, 9000);<br>
 
myRemoteLocation = new NetAddress("127.0.0.1", 9000);<br>
//make a connection
}<br>
void setup () {
// this is where you can make things happen and send values to Unity<br>
  osP5 = new OscP5 (this, 9000);
void draw () {<br>
  myRemoteLocation = new NetAddress("127.0.0.1", 9000);
OscMessage myMessage = new OscMessage ("/");<br>
}
// add a value to the message (optional)<br>
 
myMessage.add(somevalue);<br>
// this is where you can make things happen and send values to Unity
// add a bool to the message (optional)<br>
void draw () {
myMessage.add(true);<br>
  OscMessage myMessage = new OscMessage ("/");
oscP5.send(myMessage, myRemoteLocation);<br>
    // add a value to the message (optional)
}<br>
    myMessage.add(somevalue);
// this is where you get feedback in Processing<br>
    // add a bool to the message (optional)
void oscEvent (OscMessage theOscMessage) {<br>
    myMessage.add(true);
print("### received an osc message.");<br>
  oscP5.send(myMessage, myRemoteLocation);
print(" addrpattern: "+theOscMessage.addrPattern());<br>
}
println(" typetag: "+theOscMessage.typetag());<br>
 
println("### received an osc message. with address pattern"+theOscMessage.addrPattern()+" typetag " +theOscMessage.typetag());<br>
// this is where you get feedback in Processing
}</code>
void oscEvent (OscMessage theOscMessage) {
  print("### received an osc message.");
  print(" addrpattern: "+theOscMessage.addrPattern());
    println(" typetag: "+theOscMessage.typetag());
    println("### received an osc message. with address
    pattern"+theOscMessage.addrPattern()+" typetag
    +theOscMessage.typetag());
}
</source>
 
== Unity ==
== Unity ==
With just a little help from Google (or bing if you’re one of those people) you can find one of these Osc Receiver Scripts. You will also need scripts for [https://github.com/heaversm/unity-osc-receiver/tree/master/Assets/Plugins Osc and Udp].  
With just a little help from Google (or bing if you’re one of those people) you can find one of these Osc Receiver Scripts. You will also need scripts for [https://github.com/heaversm/unity-osc-receiver/tree/master/Assets/Plugins Osc and Udp].  
Line 64: Line 73:
<br>
<br>


Public values are adjustable without any coding; just use the inspector for that. <br><code>
Public values are adjustable without any coding; just use the inspector for that. <br>
public var RemoteIP : String = "";<br>
<source lang="javascript">
public var SendToPort : int = xxxx;<br>
public var RemoteIP : String = "";
public var ListenerPort : int = yyyy;<br>
public var SendToPort : int = xxxx;
public var controller : Transform;<br>
public var ListenerPort : int = yyyy;
public var gameReceiver = "Cube";</code> <br><br>
public var controller : Transform;
In the Start() function all the necessary scripts and components are connected. This will happen once you press Play. <br><code>
public var gameReceiver = "Cube";
public function Start ()<br>
</source>
{<br>
In the Start() function all the necessary scripts and components are connected. This will happen once you press Play. <br>
var udp : UDPPacketIO = GetComponent("UDPPacketIO");<br>
<source lang="javascript">
udp.init(RemoteIP, SendToPort, ListenerPort);<br>
public function Start ()
handler = GetComponent("Osc");<br>
{
handler.init(udp);<br>
var udp : UDPPacketIO = GetComponent("UDPPacketIO");
handler.SetAllMessageHandler(AllMessageHandler);<br>
udp.init(RemoteIP, SendToPort, ListenerPort);
}</code> <br><br>
handler = GetComponent("Osc");
The AllMessageHandler() function is where the Osc happens. This function unpacks the Osc signals, so you can address them in other functions. In this function you can command, whenever A happens, you want FunctionB() to be executed. <br><code>
handler.init(udp);
public function AllMessageHandler(oscMessage: OscMessage){<br>
handler.SetAllMessageHandler(AllMessageHandler);
var msgString = Osc.OscMessageToString(oscMessage);<br>
}
var msgAddress = oscMessage.Address;<br>
</source>
var msgValue = oscMessage.Values[0];</code> <br><br>
The AllMessageHandler() function is where the Osc happens. This function unpacks the Osc signals, so you can address them in other functions. In this function you can command, whenever A happens, you want FunctionB() to be executed. <br>
So you typed all that code and there are no errors but that stupid thing still won't do what it should - try Debug.Logs! Just add them everywhere! They can tell you what parts of your script are accessed after you press Play. <br><code>  
<source lang="javascript">
Debug.Log("Stakkars meg, som har skrevet 50 Debug.Logs");<br></code>
public function AllMessageHandler(oscMessage: OscMessage){
The lines will be written in the console section of Unity once they are accessed.<br>  
var msgString = Osc.OscMessageToString(oscMessage);
var msgAddress = oscMessage.Address;
var msgValue = oscMessage.Values[0];
</source>
So you typed all that code and there are no errors but that stupid thing still won't do what it should - try Debug.Logs! Just add them everywhere! They can tell you what parts of your script are accessed after you press Play. <br>
<source lang="javascript">
Debug.Log("Stakkars meg, som har skrevet 50 Debug.Logs");
</source>
The lines will be written in the console section of Unity once they are accessed.<br><br> 
[[File:50Debug_Logs.png]]
[[File:50Debug_Logs.png]]


Line 99: Line 116:
c) Add Audio variables to your script. This script will also draw a circle depending on the audio volume. <br>
c) Add Audio variables to your script. This script will also draw a circle depending on the audio volume. <br>
<source lang="javascript">
<source lang="javascript">
float x;<br>
float x;
float y;<br>
float y;
Minim minim;<br>
 
Minim minim;
AudioInput input;
AudioInput input;
</source> <br>
</source>
d) In the Setup() function add the values for the circle and create the audiotoolkit <br>
d) In the Setup() function add the values for the circle and create the audiotoolkit <br>
<source lang="javascript">
<source lang="javascript">
size (320, 240);<br>
size (320, 240);
smooth();<br>
smooth();
stroke (255, 25);<br>
stroke (255, 25);
noFill ();<br>
noFill ();
// Set start position<br>
 
x = 0;<br>
// Set start position
y = 20;<br>
x = 0;
minim = new Minim (this);<br>
y = 20;
input = minim.getLineIn (Minim.STEREO, 512);<br>
 
minim = new Minim (this);
input = minim.getLineIn (Minim.STEREO, 512);
background (0);
background (0);
</source> <br>
</source>
e) The draw() function will be the container of the circle’s size. It will only send the values to Unity if the circle is big enough. <br>
e) The draw() function will be the container of the circle’s size. It will only send the values to Unity if the circle is big enough. <br>
<source lang="javascript">
<source lang="javascript">
void draw () {<br>
void draw () {
int i =0;<br>
int i =0;
float dim = input.mix.level () * width;<br>  
float dim = input.mix.level () * width;<br>  
if (dim >= 80){System.out.println("Loud enough: "+dim);<br>
 
OscMessage myMessage = new OscMessage("/");<br>
if (dim >= 80){System.out.println("Loud enough: "+dim);
myMessage.add(dim);<br>
  OscMessage myMessage = new OscMessage("/");
myMessage.add(true);<br>
    myMessage.add(dim);
oscP5.send(myMessage, myRemoteLocation);}
    myMessage.add(true);
</source> <br>
  oscP5.send(myMessage, myRemoteLocation);}
</source>
f) When you are ready, press Play. <br> <br>
f) When you are ready, press Play. <br> <br>
'''2. In Unity:''' <br>
'''2. In Unity:''' <br>
Line 133: Line 154:
b) Open up the Osc Receiver Script.  Define a new private variable. <br><br><source lang="javascript">
b) Open up the Osc Receiver Script.  Define a new private variable. <br><br><source lang="javascript">
private var moveSpeed : int = 0;
private var moveSpeed : int = 0;
</source> <br>  
</source>  
c)  Create a new public function.<br>
c)  Create a new public function.<br>
<source lang="javascript">
<source lang="javascript">
Line 140: Line 161:
moveSpeed = msgValue;
moveSpeed = msgValue;
}
}
</source> <br>
</source>
d)  In the Update() function add <br>
d)  In the Update() function add <br>
<source lang="javascript">
<source lang="javascript">
var go = GameObject.Find(gameReceiver);<br>
var go = GameObject.Find(gameReceiver);
go.transform.Translate(0, moveSpeed, 0);  
  go.transform.Translate(0, moveSpeed, 0);  
</source> <br>
</source>
e) In the AllMessageHandler() function add <br>
e) In the AllMessageHandler() function add <br>
<source lang="javascript">
<source lang="javascript">
Line 152: Line 173:
   MoveObject(msgValue);
   MoveObject(msgValue);
   break;}
   break;}
</source> <br>
</source>
f) When you are ready, press Play.
f) When you are ready, press Play.



Latest revision as of 11:05, 27 July 2016

On 1 Computer

1 Computers.png

1. Open Unity and Processing.
2. In both programs the Remote IP should be set to 127.0.0.1
3. The SendToPort in Processing equals the ListenerPort in Unity. You will have to adjust the values manually. Usually port values between 7000 and 15000 are not taken, so you can use them for your OSC connection. If it is not working try changing the ports. Do not use the same value twice at the same time, if you have more than one connection running.

On 2 Computers

2 Computers.png

1. Make sure you are connected to the same Wifi. If it does not work there might be an active firewall disturbing your work, so turn that off if you have to.
2. Open Unity and Processing.
3. In Unity fill in the IP of the Computer that is running Processing.
4. The SendToPort in Processing equals the ListenerPort in Unity. You will have to adjust the values manually. Usually port values between 7000 and 15000 are not taken, so you can use them for your OSC connection. If it is not working try changing the ports. Do not use the same value twice at the same time, if you have more than one connection running.

Processing

In Processing you need to install a library. Click on ToolsAdd Tools – Choose Libraries – Search for oscP5 and install it.

This is the basic processing script for sending Osc data (it just sets up the connection, actions can be added):

//load libraries
import netP5.*;
import oscP5.*;

OscP5 oscP5;
NetAddress myRemoteLovation;

//make a connection
void setup () {
  osP5 = new OscP5 (this, 9000);
  myRemoteLocation = new NetAddress("127.0.0.1", 9000);
}

// this is where you can make things happen and send values to Unity
void draw () {
  OscMessage myMessage = new OscMessage ("/");
    // add a value to the message (optional)
    myMessage.add(somevalue);
    // add a bool to the message (optional)
    myMessage.add(true);
  oscP5.send(myMessage, myRemoteLocation);
}

// this is where you get feedback in Processing
void oscEvent (OscMessage theOscMessage) {
  print("### received an osc message.");
  print(" addrpattern: "+theOscMessage.addrPattern());
    println(" typetag: "+theOscMessage.typetag());
    println("### received an osc message. with address
    pattern"+theOscMessage.addrPattern()+" typetag
    +theOscMessage.typetag());
}

Unity

With just a little help from Google (or bing if you’re one of those people) you can find one of these Osc Receiver Scripts. You will also need scripts for Osc and Udp.

1. Import the scripts to the Assets folder using drag&drop. If you’re working on a real actual project it might be best to create a folder for all your scripts.
2. Create an empty game object (right click in the hierarchy) and rename it, so you know what it is.
3. From the assets you drag the scripts onto the empty game object.
4. Do not forget to adjust the IP, and the port in the inspector.
IPandPortintheOSCReceiver.png
5. In the Inspector you see the public variable Game Receiver within the Osc Receiver Script. Drag the object you want to manipulate from the hierarchy onto the Game Receiver value. By default this value is called “Cube”.
GameReceiver.png
6. If you haven't done it already - Safe! Ctrl+S on Windows, Macs probably do that differently, but you will know that better than me.

If all this code is completely new to you and you only understand trainstation – here is an explanation for the whole OSC Receiver Script:

Public values are adjustable without any coding; just use the inspector for that.

public var RemoteIP : String = "";
public var SendToPort : int = xxxx;
public var ListenerPort : int = yyyy;
public var controller : Transform;
public var gameReceiver = "Cube";

In the Start() function all the necessary scripts and components are connected. This will happen once you press Play.

public function Start ()
{
 var udp : UDPPacketIO = GetComponent("UDPPacketIO");
 udp.init(RemoteIP, SendToPort, ListenerPort);
 handler = GetComponent("Osc");
 handler.init(udp);
 handler.SetAllMessageHandler(AllMessageHandler);
}

The AllMessageHandler() function is where the Osc happens. This function unpacks the Osc signals, so you can address them in other functions. In this function you can command, whenever A happens, you want FunctionB() to be executed.

public function AllMessageHandler(oscMessage: OscMessage){
 var msgString = Osc.OscMessageToString(oscMessage);
 var msgAddress = oscMessage.Address;
 var msgValue = oscMessage.Values[0];

So you typed all that code and there are no errors but that stupid thing still won't do what it should - try Debug.Logs! Just add them everywhere! They can tell you what parts of your script are accessed after you press Play.

Debug.Log("Stakkars meg, som har skrevet 50 Debug.Logs");

The lines will be written in the console section of Unity once they are accessed.

50Debug Logs.png

Example: Make an object jump on clapping


1. In Processing:
a) Install an additional library for sound. Click on ToolsAdd Tools – Choose Libraries – Search for Minim and install it.
b) To your imported Libraries add

import ddf.minim.*;

c) Add Audio variables to your script. This script will also draw a circle depending on the audio volume.

float x;
float y;

Minim minim;
AudioInput input;

d) In the Setup() function add the values for the circle and create the audiotoolkit

size (320, 240);
smooth();
stroke (255, 25);
noFill ();

// Set start position
x = 0;
y = 20;

minim = new Minim (this);
input = minim.getLineIn (Minim.STEREO, 512);
background (0);

e) The draw() function will be the container of the circle’s size. It will only send the values to Unity if the circle is big enough.

void draw () {
 int i =0;
 float dim = input.mix.level () * width;<br> 

 if (dim >= 80){System.out.println("Loud enough: "+dim);
  OscMessage myMessage = new OscMessage("/");
    myMessage.add(dim);
    myMessage.add(true);
  oscP5.send(myMessage, myRemoteLocation);}

f) When you are ready, press Play.

2. In Unity:
a) Create an object in Unity. Drag this object onto the gameReceiver of your Osc Receiver Script. This object is now the gameReceiver and will behave as you define in the Osc Receiver script.

b) Open up the Osc Receiver Script. Define a new private variable.

private var moveSpeed : int = 0;

c) Create a new public function.

public function MoveObject(msgValue) : void
{
moveSpeed = msgValue;
}

d) In the Update() function add

 var go = GameObject.Find(gameReceiver);
  go.transform.Translate(0, moveSpeed, 0);

e) In the AllMessageHandler() function add

switch (msgAddress){
 default:
  MoveObject(msgValue);
  break;}

f) When you are ready, press Play.

Example: Use your mouse position to Control Objects

S O O O O O O O O O N