If I had a page having the noindex tag <meta name="robots" content="noindex" />, I know that the page won't be shown on search results.
But will the links on the page still be crawled?
If I want them to be crawled, can I set something like <meta name="robots" content="noindex, follow" />. Will it work?
The noindex tag <meta name="robots" content="noindex" /> is used to prevent most search engine web crawlers from indexing a page on your site.
The crawlers will not index the page using content="noindex, follow" tag but will crawl through the rest of the pages.
You can find more details here: http://www.robotstxt.org/meta.html
