I have a USB3 to HDMI adapter based on UFX7000 SMSC chipset. It has a driver in the Linux kernel, but it makes a framebuffer device:
usb 2-1.1.1: SMSC UDX USB device /dev/fb1 attached. 800x600 resolution. Using 940K framebuffer memory
Is there an easy way to use that fb1? xrandr won't see what's attached to it, so it's either making a fixed multi-display fixed x.org config or maybe (is that possible?) starting xephyr/other X server/specific app on /dev/fb2.