4

Edit: I am not getting enough response for this question may be i am not able to explain properly. Let i try to explain background to this question.

Every screen sharing program there are two parties. Server and Viewer.

Server

Role of server

  • capture desktop screen
  • compare capture screen with previous sent screen
  • calculate dirty region convert dirty region to byte array
  • send byte array to tcp socket to viewer

Viewer

Role of Viewer

  • receive bytes from server
  • convert bytes to image
  • Draw image to viewer picturebox/Panel

Problem to tackle

  • There are many duplicate region/area server sends which is previously sent to viewer 10/20 frames back.I want to avoid this using hashing.

Below is my analysis

I am studying how screen transfer application like Team Viewer,Any desk works and identifying How they works well in very low bandwidth internet connection.

While studying i came across with Screen sharing app Any Desk. Which use xxHash to hash images and send image byte data along with his hash code.

I used network speed monitor on my mobile internet and started application.

I obverse that performing same activity (e.g open/close same directory,minimize and maximize any application) on desktop multiple times. app does not send any data/Regions to client except first request. I assume that it sends only hash code to client if it finds duplicate image/Dirty Region hash code.

My question is how feasible this method, because on client side. we need to store hashcode along with image bytedata in memory.How this apps are memory efficient. or is there any other way they doing it.

Edit

Screen sharing program does not send whole screen/Region each time.it just send only difference/Dirty reigon over the internet.along with if user has very bad internet connection it stores bytearray of each frame/Region along with hashcode.Send only hashcode if hascode matches with difference frame/region bytearray

Server Code assumption

     List<uint> hashlist = new List<uint>();
        byte[] imagesBytestosend = GetImageBytes();

        uint hash = XXHash.XXH32(imagesBytestosend);

        if (!hashlist.Contains(hash))
        {
           hashlist.Add(hash);
          SendAllBytesData(imagesBytestosend);
        }
        else
        {
           SendOnlyHasCode(hash);

        }

Client Code assumption

    Dictionary<uint, byte[]> frameData = new Dictionary<uint, byte[]>();
        bytes[] imageBytes = null;
        if(frameData.ContainsKey(receivedHash))
        {
        imageBytes=frameData[receivedHash];
        }
        else
        {
        imageBytes=receivedBytes;;
        frameData.Add(receivedHash, receivedBytes);
        //My Question is how this is memory efficient to store frame bytes with his hash code in memory
        }
        DrawToPictureBox(imageBytes);
Azar Shaikh
  • 445
  • 9
  • 26

2 Answers2

0

If the image is still open, then it is still in the memory at the client side. So the only overhead is the hashcode, which is quite small. If anything, it is efficient in the scenario you described. If the hashcode weren't stored, then the same image (which already is in memory) would be sent over again, thus wasting bandwidth and leading to related performance issues. However, since the hash is saved, the client only has to redraw the same image from the bytes in memory.

Of course, when you close that image, the memory associated with it is probably released soon after. As such, if you were to open the same directory after some time, the image date, as well as the hashcode, would have to be sent over again.

AAA
  • 3,520
  • 1
  • 15
  • 31
  • Storing image byte array along with hashcode.How it would be memory efficient. – Azar Shaikh Jun 05 '19 at 05:53
  • Every image drawn to the screen is stored in memory, right? So it's not like it's stored twice. It's only the hashcode stored. This is much better than storing the same image multiple times – AAA Jun 05 '19 at 14:59
  • Ok let me clarify you.Pc1 taking screenshot calculating difference and sending to pc2. Pc2 drawing image coming from pc1 to a picturebox.pc2 picture box overwritten with new images comming from pc1.If I need to use hashcode pc2 need to store each frame buffer with his hashcode to memory to redraw from memory. – Azar Shaikh Jun 05 '19 at 15:52
  • If one frame weight is 100 kb and I need to store 1000 frames it's 100mb application memory usage – Azar Shaikh Jun 05 '19 at 15:54
  • It's the image that is stored, not a video. Regardless of using remote application or not, every image drawn to your screen at any time is stored in memory. That's the only way it's drawn to the display. That's why you usually want to to have a GPU with dedicated memory to handle graphics. – AAA Jun 05 '19 at 20:35
0

It sounds like you have a fundamental lack of understanding of how Screen Sharing works. In Windows for instance, we do not detect changes in the screen buffer. Instead we rely on Windows marking regions of the screen as invalid.

https://learn.microsoft.com/en-us/windows/desktop/gdi/the-update-region

You might notice that when you enable screen sharing, the Windows Theme drops from Aero down to Basic themes.

Aron
  • 15,464
  • 3
  • 31
  • 64
  • This might be true for your typical windows application, but you cannot rely on win32 api on other os-ses. Even more, this technique will fail for most hardware accelerated graphics applications. – Stefan Jun 10 '19 at 07:59
  • 1
    @Stefan That is why most screen sharing applications disable most of the accelerated features, forcing you out of Windows Aero. – Aron Jun 10 '19 at 08:00
  • Yes I know, my point is; you are assuming OP's use case doesn't require this functionality. For all you know this would be some part of a Google chrome cast like application, which would miserably fail for video or 3d overlays/applications. The point is, your solution only works within a small specific subset of all possible use cases. – Stefan Jun 10 '19 at 08:12
  • @Stefan As would almost every single solution under the sun. No generic solution to your problem exists. It is about optimization. Since OP was "studying" software such as "AnyDesk" I assumed that he wasn't building Google Stada, nor a FMV streaming system. – Aron Jun 10 '19 at 08:19
  • @Stefan I also made the assumption that he wasn't trying to stream FMV due to the fact that he was using C#. Which, having a non-deterministic memory model, make near impossible. – Aron Jun 10 '19 at 08:21
  • Multimedia streaming is perfectly doable in c#, and that has nothing to do with the memory model. – Stefan Jun 10 '19 at 08:32