To be honest I think this is a bit of a grey area. Personally I think that if you alter the functions of those buttons your app will get rejected from the Apple iOS App Store Review process for breaching guideline 10.5. However I also don't believe that Apple will test your App with headphones unless you specifically state that headphones are required for your app to work. Though if your app needs headphones to access certain functionality and you didn't tell Apple so they didn't test it and they found out this could possibly get your account banned for dishonesty.
So in all honesty based on the review guidelines I'd say this can't be possible for the reason already stated and I also think that you'd need to access some private APIs to actually achieve this. So would also be rejected under 2.5.
2.5 Apps that use non-public APIs will be rejected
10.5 Apps that alter the functions of standard switches, such as the Volume Up/Down and Ring/Silent switches, will be rejected
So if you are going to continue with this I'd tread very carefully about what you (1) do to implement it and (2) what you tell Apple when it goes into the review process. If it was me I'd tell Apple that there is functionality behind those buttons.
However I really don't think this would get past the review process for 2.5 no mind reaching 10.5