Well, to answer the question you're asking, you need to install DNS on that server before you can configure DNS on that server. If you need to do this on Windows 2008, it is configured as a windows 2008 role. So just choose to add a role from the Windows 2008 management tools, select DNS and go from there.
There will still be some work to set DNS up before you can just add your PTR record - you will need to create a forward zone to manage your domain name and associated records, and should really be creating PTR/reverse lookup records for all hosts.
However, Before you do that consider this: your server must presumably already be pointing to a DNS server somewhere, which may be one owned by your ISP/webhost and just refers to the IP addresses you use generically, or it may be one that manages a block of IPs and a domain name for your Internet presence specifically - it is this DNS server that needs a PTR record configuring for your mail server, so investigate this before trying to install and configure your own DNS.