Edit:
Visual Studio 2015 Update 1 changes the feedback icon to an understated black and white one so no more smiley! It's not mentioned in the release notes.

Original answer:
As in OPs answer, this icon is specified in this registry key:
HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\14.0_Config\MainWindowFrameControls{F66FBC48-9AE4-41DC-B1AF-0D64F0F54A07}
if you delete the key, Visual Studio recreates it, but if you invalidate the "Package" value of the registry key and restart Visual Studio then the smiley icon is gone:

However, when you install a Visual Studio update (eg SSDT, Resharper) the installer restores the package value, and the smiley is back. So I have created a registry file like this to run when the smiley reappears:
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\14.0_Config\MainWindowFrameControls\{F66FBC48-9AE4-41DC-B1AF-0D64F0F54A07}]
@="Feedback Button"
"Package"="{00000000-AA51-43B1-97EE-509A33B681F3}"
"DisplayName"="#1001"
"ViewFactory"="{060EAB95-139E-407D-BEDC-CC2B7A9B39D4}"
"ViewId"=dword:00000064
"Alignment"="TitleBarRight"
"Sort"=dword:00000064
"FullScreenAlignment"="MenuBarRight"
"FullScreenSort"=dword:00000064
This doesn't seem to affect startup time or stability, but there are no guarantees, registry changes are bad mmm, etc.