You can run an app from a folder without registering any exe or dll.
A filecopy would work.
If the wan is available you could maybe push install new versions using AD server software such as SMS. Maybe this is a pc in a warehouse or nuclear facility and totally air gapped.
The way I would suggest approaching this is to filecopy the app to the user's appdata. Exe, dll config just file copy it all. Into a v1 folder.
When the user gets a new usb version you filecopy that to a v2 folder.
Restarting the app is then a matter of running the exe from the highest version folder. Which means a small loader app or update the registry with current location so when they start xxx it will work out location from there. Which might not be practical if they can't update the registry.
You can the approach more sophisticated using mef.
Make the entry point mainwindow very simple and that hosts content from dll which are dynamically discovered. They can be discovered from the highest version folder.
Resource dictionaries can be loaded from there and either delivered uncompiled as raw xaml files or in a usercontrol library.
Hence your main functionality would move into say a usercontrol library.
The actual exe would have very little in it.
You might want to consider full on PRISM here which allows you to dynamically load the shell and everything. If you've not done this before it's perhaps worth mentioning there'd be a faire amount of PRISM knowledge required to get that working.