The answer to all such questions is in the way iptables
processes rules, ie first dispositive match wins. That means you write your most specific rule first, moving out to the least specific; eg:
iptables -A INPUT -p tcp --dport 12001 -s a.b.c.d -j ACCEPT
iptables -A INPUT -p tcp --dport 12001 -j REJECT
# and similarly for UDP, then...
iptables -A INPUT -j ACCEPT
The first allows traffic from the specified favoured address (here, a.b.c.d) to the specified port (12001); the second refuses all other traffic to that port; the third allows everything else.
As a result, rule 2 doesn't have to contain an exception for the approved traffic, because rule 1 has already allowed it; the allowed traffic will never see rule 2, so won't be bothered by it. Similarly, rule 3 doesn't have to deal with refusing most of the traffic to port 12001, because rule 2 has already dealt with that.