Recognizable problem.
Usually config fields with sensitive information like passwords change a lot less often than non-sensitive configuration fields. A possible solution is to split the config in two parts:
- Config that's environment-specific but doesn't contain sensitive information. I would advise you to keep these files together with your source code and if possible, generate the files and automatically upload then to your configuration store (S3 in your case) at build time. They must be versioned and tied to the version of your application.
- Config that contains sensitive information. Looking at the question, not all team members are allowed to read/write this information. You could store these in S3 with specific access rights so that only authorized members can access them. You would need a mechanism to join the files back together at deployment, or change the app to read from different config files.
However, this will only solve part of your problem. The ops guys will still need to perform changes when sensitive config keys change. Whether this is acceptable depends on how often sensitive config keys change.
An alternative to S3 could be to run a private Git repository (AWS's CodecCommit, for example). You'd have better version control and easier access for the devs to perform changes, since you're already using Git. You'll still have to fix the split access rights between dev and ops, or let that go (since DevOps is about trust and cooperation, that might be a good idea). You could apply a similar pattern here as I described above.
Another solution could be to move the configuration of sensitive values from property files to the system configuration. When you already use a provisioning system like Puppet or Chef, this will feel natural for the ops guys. Or set all sensitive values like passwords as environment variables and have the app read it as system properties.
Hope this helps!