I am not sure what use cases you want to prohibit exactly, but in general your app can check each incoming request and decide if and how it wants to react to it.
For example you will always get the user ID of who sent the slash command or message to the bot. You can use that to filter our users that should not have access.
Update
To restrict your app the the app channel you need to do the following:
When receiving a request from the user, first open a direct message channel to the user from the bot user. That will always give you the channel ID of the app channel.
Then reply with a direct message in that app channel
or alternatively check if the received request is from the app channel and ask the user to only talk in app channel if it is not.
See also this answer on how this works in detail.