3

I am trying to continuously send small UDP packets (containing only couple of bytes) from Hololens 2 to a computer over wifi. In my case it is necessary to transfer these data at a stable rate of 60fps - without any dropouts. This works as expected, but I have encountered a strange problem - incoming data start "lagging" heavily every 5 minutes. This lagging happens for a couple of seconds and then it reverts back to stable and continuous stream. Unfortunately this is unacceptable for my use-case and I am hoping to get rid of this issue. This gif shows lagging of incoming udp data in real-time (each datagram contains frame number that is visualized in graph).

I have been trying to analyze this problem for quite some time, but so far I haven't figured out what is causing it. Nevertheless, I can provide these conclusions from my tests:

  1. Wifi signal isn't a problem. I have a wifi access point placed directly next to a Hololens 2 - providing excellent signal. Whole setup consists of my laptop that is connected directly to Cisco access point (through ethernet connection) and Hololens 2 which is connected to its wifi. There is no internet access on this network - it is used just for local networking. Also there isn't any network interference playing a role in this problem as I have tried this at various locations with the same results. I have also tried adding one more computer onto this wifi to see if I could get a stable stream from it - both Hololens 2 and this computer were streaming similar data to my laptop. During the phases where Hololens data were lagging, data from this computer were stable without any issue.

  2. Application itself running on Hololens 2 has a stable rate of 60fps without any frame drops. I run tests with simple app created using Unity and also with native UWP app (both behaved the same in terms of networking). I have also observed the same results with different socket implementations - both System.Net.Sockets and Windows.Networking.Sockets performed the same.

  3. Interestingly, this problem doesn't occur when I connect to Hololens 2 directly through ethernet. I have successfully tested this with usb-c to ethernet adapter which provided stable data transfer (without any lagging). Therefore it is clear that lagging is present only when using wireless network adapter on Hololens 2.

  4. I am definitely no network specialist, but I have tried to examine packets coming in to my laptop network interface through Wireshark, but there wasn't any "suspicious" activity happening during these lagging periods. Out of curiosity I have also captured packets directly on Cisco access point and have created following graphs in order to visualize what was happening there.

  • This graph shows arrival time of all packets (y axis represents time in seconds, x is packet id). The middle area is clearly suffering from mentioned lagging.

  • This graph shows only filtered packets that belong to my udp communication - all other packets are displayed with 0 time in order to visually identify packets used for different (unknown) communication. You can clearly tell that majority of other requests happened exactly in those problematic areas. Unfortunately I don't know what those requests are as the capture output from Cisco access point seemed rather cryptic to me. All I could tell is that majority of those requests were sent from Hololens 2 to Cisco access point.

Based on these observations I would say that every 5 minutes is Hololens 2 doing something in the background that causes these network problems. It influences only wifi network interface. Even though I haven't tried holographic remoting or spectator view, I believe they must suffer from same lagging. Please does anyone how to resolve this problem? Since Hololens 2 doesn't have control over firewall or services, I am feeling kind of stuck here and was hoping that some Hololens developer could help me. Thanks.

monty_python
  • 35
  • 5
  • Could you check for updates in Settings to see If there is a system update available for HoloLens 2? So that we can avoid some known issues. Windows Holographic, version 2004 fixed a bug where Wi-Fi streaming to HoloLens got disrupted periodically, for more information please see:[Quality improvements and fixes](https://learn.microsoft.com/en-us/hololens/hololens-release-notes#quality-improvements-and-fixes) – Hernando - MSFT Apr 19 '21 at 06:56
  • @Hernando-MSFT thanks for info. Yes - my system is updated to the latest version (no update is available when checking for updates). However the mention of `SetSocketMediaStreamingMode` function in release notes caught my attention as it sounds like something that could possibly help in this scenario. I will try to implement this function call to see if it might change current behavior. – monty_python Apr 19 '21 at 10:25
  • Unfortunately using `SetSocketMediaStreamingMode` didn't help in this case - the issue persists. – monty_python Apr 19 '21 at 11:55
  • As a test, could you please download a complete DatagramSocket UDP sample app from this link: [DatagramSocket sample](https://github.com/microsoft/Windows-universal-samples/tree/master/Samples/DatagramSocket), and then deploy it to your HoloLens2? Do you see the same behavior? This will help narrow down if it's an issue with your code. – Hernando - MSFT Apr 20 '21 at 06:19
  • Sure - I have just tried this sample and it behaved exactly the same... I have sent UDP data from my laptop at the rate of 60fps (each datagram containing frame number). DatagramSocket sample app then echoed these data back to me - allowing me to nicely visualize incoming stream (which was lagging every 5 minutes). – monty_python Apr 20 '21 at 08:04
  • Could you please provide an MVCE(stackoverflow.com/help/minimal-reproducible-example) if you have made some modifications on the sample DatagramSocket to transfer these data at a stable rate? So that we can locate the issue or find a solution. – Hernando - MSFT Apr 21 '21 at 09:50
  • I haven't done any modifications to it whatsoever. I have compiled unchanged DatagramSocket sample and utilized its echo functionality to test this. However such test also required custom application running on my laptop which was sending UDP packets to DatagramSocket sample running on Hololens2. Setup worked like this: UDP packets were streamed from my laptop at rate 60fps (each containing frame number), Hololens2 was echoing them back to my laptop where I was visualizing incoming stream (by graphing frame numbers). I guess what you need is app that was running on my laptop, is that right? – monty_python Apr 22 '21 at 09:00
  • Yes, if you are willing to provide the test app for us to reproduce, please remove the privacy information from your app and open a support ticket through this link: http://aka.ms/hololenssupport – Hernando - MSFT Apr 22 '21 at 09:50
  • All right, I have created a support ticket, in case it might help this is the ticket id: 121042225001854 – monty_python Apr 22 '21 at 12:01
  • Having done networking with the hololens 1, this kind of packet drop over udp + wifi is common... I dont have a solution for you but we built the application to be robust against this kind of loss. You may want to test this in a room far away from any other RF signals and a wifi router dedicated to just the hololens and your pc. If your cisco ap is shared with other network users, if the wifi buffer is full the default behavior of the ap is to drop all packets that dont fit in the buffer. There is also no way to ensure that a packet which leaves the hololens first will be received first. – aCuria May 18 '21 at 08:14

0 Answers0