Yes, the name servers that are serving up the A, MX, etc records need to know that they are authoritative, so they must also have the NS records. You should configure one as a Master and the other as a Salve (if you have more than two the rest should all be Slaves too).
The other answer has a lot of good information, but I don't think he ever actually answers your question.
If you're running BIND, your Master DNS server should have an entry like this in named.conf
:
acl "mysecondaries" {
12.34.56.78;
};
zone "example.com" {
type master;
file "master/example.com";
allow-transfer { "mysecondaries"; };
};
On the Slave server something like this:
zone "example.com" {
type slave;
file "slave/example.com";
masters { 1.2.3.4; };
};
The actual record (make this file only on the Master) should look like this:
example.com. IN SOA ns1.example.com. root.example.com. {
2012082401 ; serial
10800 ; refresh (3 hours)
3600 ; retry (1 hour)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS ns1
NS ns2
MX 10 mail
ns1 A 1.2.3.4
ns2 A 12.34.56.78
mail A 2.34.5.67
www CNAME park.domain.net
@ CNAME park.domain.net
While I'm at it. There are Free secondary/slave DNS services so you don't need to operate two of your own. Many personal and small websites are setup this way. I use Afraid.net and Buddyns.com myself.
To add them, add the following to your named.conf on your Master server:
acl "afraid" {
174.37.196.55;
};
acl "buddyns" {
173.244.206.26;
174.136.99.149;
};
And change your allow-transfer directive for each zone in your named.conf
on your Master:
allow-transfer { "afraid";
"buddyns";
"mysecondaries"; };
And add the relevant ns
records to your zone files:
NS ns2.afraid.org.
NS b.ns.buddyns.com.
NS c.ns.buddyns.com.
There are various ways to get really fancy with these configurations (if the above seems really manual, it is; but that also makes it fairly easy). Unless you're parking a ton of domains you really don't need to get into anything more complicated.