0

I am looking for a solution to share the screen from a mobile AR app (ARKit or Unity AR Foundation). The screen needs to be shared to a browser on the desktop and it should be possible to draw lines on the screen from the browser using the mouse in the AR environment that can be seen on the mobile app which is sharing the screen.

After some investigation there does not seem to be a viable solution to truly share the same AR instance with browser/mobile as you can do with 2 mobile devices. There should however be some sort of work around possible as it can be done with Vuforia Chalk AR.

Here is a GIF showing how it works:

AR Drawing demo

  • Sharing a video seems to be possible
  • Specifically trying to figure out how the line is drawn from the browser and then displayed on the mobile AR app

How can you achieve the same functionality with open source alternatives or Unity and custom code (No Vuforia is possible)? Looking for a tutorial or some directions to how this can be implemented.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
0xFlem
  • 1
  • 1
  • To share your Mobile Camera on browser, you can use Sockets. At first you need to convert every frame of your webcam image into a byte array (with a reasonable frame rate) and send them to browser (on a local network or server on internet) by network transitions protocols like TCP or UDP (UDP is recommended for video transition). an example of UDP server is found https://stackoverflow.com/questions/20038943/simple-udp-example-to-send-and-receive-data-from-same-socket – MohammadReza Arashiyan May 24 '20 at 09:41
  • Thanks for the answer, I have found some example of this, for my use case I'll probably need to share the screen though if I want the AR drawings to show on the video in the browser. The tricky thing is however to be able to draw lines from the browser I am streaming to that will also show on the phone. Maybe it would need to send the coordinates somehow of the lines back to the phone and the AR engine on the phone would draw the lines on the scene, not sure how this could work with depth though. – 0xFlem May 25 '20 at 11:18
  • very nice, please share your experience here – MohammadReza Arashiyan May 25 '20 at 11:35

0 Answers0