The Toolbox can still be slow when closed. I notice the statement in the STATUS BAR at the bottom sometimes stating "updating toolbox from assembly XXX" or something like that. And it is closed, not collapsed or hidden!
You can disable opening XAML in design view, which can speed up things a lot, but it does mean that you loose intellisense.
For me it was worth it at some point, and I don't need it always.
To disable the WPF designer mode for XAML, right-click the .xaml file, click "Open With", select XML editor (and consider SET AS DEFAULT).
If you set it as the DEFAULT way, you can always change your mind and open specific files in design view again by selecting OPEN WITH, and then the AUTOMATIC EDITOR at the top, or set it back as the DEFAULT mode.
You will notice that whenever you use the DESIGNER view again, things are A LOT slower for WPF XAML files.