Most settings will not be re-read if you change them at runtime. So Django will not recognise the changes you make.
This is due to the fact that Django is just normal Python code. It isn't like a server that is monitoring your code - it is just part of your code.
In some cases, parts of Django code might respond to changes in settings, because they might do 'settings.DEFAULT_FROM_EMAIL' every time mail is sent, for example.
But if Django processes the setting in any way, like it has to do for INSTALLED_APPS, it isn't going to notice you changed something and re-do the processing.
Which settings are safe? Well, the docs are saying "none are safe", because it might change in the future. Django might save a copy of any setting for some reason, or do some processing.
Changing INSTALLED_APPS could never be made to work, because it alters which modules are imported. There is simply no way that Django could work around the way that Python works at this level - it would need to be able to 'unimport' modules, which is basically impossible (the only way is to restart the process), and there are other problems associated with cross-app links.