I want to be able to build something similar to the now defunct vr program Envelop. It allows you to interact with your Windows 10 windows in a VR space https://www.youtube.com/watch?v=XPCq089MQDI
From reading on wikipedia i see that windows write to an off screen buffer provided to them by Window's DWM
https://en.wikipedia.org/wiki/Desktop_Window_Manager
It looks like this offscreen buffer is hanging out somewhere down near the GPU ram. So it seems like I should be able to get reference to that spot in memory and draw a 3d mesh with it. Is that possible? How do I find the reference?
Looking at the DWM api it looks like you can request Windows to draw a copy of another app's contents into your draw area, but they mention specifically that you can't use it to a 3d thing since its windows doing the drawing and it does it in 2d
https://msdn.microsoft.com/en-us/library/windows/desktop/aa969541(v=vs.85).aspx
How can I use windows textures in my 3d app? Also, if there is a better approach using universal apps, or virtual desktops or some such thing, I am all ears.