You can host your DNS with the company you purchased your DNS from, or point the name servers at your VPS provide and use their VPS.
In general I would avoid running a commodity service like DNS on my own server. There's nothing to gain doing that for most people, it's a little extra load and maintenance.
However, if the VPS provider provides dedicated DNS servers, that's an ok option. Then you would consider the features and reliability of the VPS provider DNS servers and your DNS registrar servers / service.
As @ceejayoz says, you can host DNS anywhere. I host my DNS on CloudFlare free plan. Super reliable, super fast. I've used Route53 in the past, which is good and costs
In general, for a beginner, I suggest you host your DNS on your domain registrar, because it's likely to be fast enough and reliable enough, but most importantly it's simple. Once you get everything working you can consider your VPS provider DNS, or better yet, CloudFlare / AWS Route53.