GMU:Max Neupert/StreamGem2Android

From Medien Wiki

Goal: Stream to Android

Goal: streaming the Gem-window of a Linux Desktop to a VR-headset running on an Android phone.

This page documents different possible approaches and hopefully will show how it's done when successful.

My phone has a resolution of 1920 × 1080 pixels

Unsucessful Approaches

Apps

If you are on Windows, you have some choices for apps, that may or may not come with their own limitations:

Using x11vnc

Use xwininfo to find out the ID of the window to be shared, then use x11vnc -id {id number here} to share only one window to the vnc client.

However cardboard-vnc-free is doing too much. First of all, it controls the moise pointer on the server (as a VNC client should) but I don't need that. There is an option "Viewer" in the connection settings that will diable this and lower the latency from about 3 seconds to 1 second.

cardboard-vnc-free is also adding two viewports to the shared image, to fake a 3D. That's a show stopper, because I want to use the stereo mode in GEM.

Gemwin remote host

It's possible to open the Gemwwin on a remote host with the [remotehost( message to [gemwin] (see gemwin-help) However, there is no further explanation of that feature and I haven't seen any discussion on pd-list or here about it.

With an X Server running on the Android.

I can see the Pd gui on my phone with $ export PULSE_SERVER=tcp:{local IP}:4712 metacity & puredata

the test audio patch makes noise on the computer.

It (expectedly) doesn't allow me to open a GEM window

error: GEM: X server has no OpenGL GLX extension
error: GEM: Error creating const context
error: GEM: A serious error occured creating const Context 
error: GEM: Continue at your own risk!
error: GEM: X server has no OpenGL GLX extension
error: GEM: Unable to create window
error: [gemwin]: no window made
error: GEM: Create window first!
error: JACKerror: SuspendRefNum error
error: JACKerror: JackClient::Execute error name = pure_data_2
error: JACK: server shut down

ffmpeg

Best overview on streaming with ffmpeg is to be found https://trac.ffmpeg.org/wiki/StreamingGuide#Pointtopointstreaming

udpvideo lite/ffmpeg

Since streaming to a VR-Headset with minimum latency is pretty much what you want with a live drone video this abandoned freeware app is doing exactly that: udpvideo lite

Then run on the desktop: ffmpeg -f x11grab -video_size wxga -r 25 -i :0.0 -f mpjpeg udp://192.168.1.2:5678

Successful

OBS/VLC

It works in OBS + VLC with this method https://obsproject.com/forum/resources/obs-studio-send-an-udp-stream-to-a-second-pc-using-obs.455/ but has a delay of a couple of seconds.