Update the files Build Action to Content using Visual Studio (e.g. right click, properties)

The Publish tasks will still transform the files, so we need to tell MSBuild, that we do not want to transform those files when publishing.
This can be achieved by passing the following parameters into MSBuild:
/p:ProfileTransformWebConfigEnabled=false /p:MarkWebConfigAssistFilesAsExclude=false
If you are working within Visual Studio, you test this behavior by adding these properties to a folder publish profile PublishProfile.xml
<!-- Disable Web.config Transforms -->
<ProfileTransformWebConfigEnabled>false</ProfileTransformWebConfigEnabled>
<MarkWebConfigAssistFilesAsExclude>false</MarkWebConfigAssistFilesAsExclude>