0

if i understand it right, you can only tell google to crawl or not crawl pdf files via robots.txt. i want google to crawl the files, but not list them in the search results pages.

is this possible?

mostwanted
  • 1,549
  • 3
  • 13
  • 21
  • 1
    Ok, so what would google do with the information? Google: ah! here's a Pdf file. Nice .. but ... what do u want me to do with this? I provide search results .. and u're asking me to NOT list this file in any search results... er..... – Pure.Krome May 12 '12 at 12:55
  • @Pure.Krome: if its a pay for download resource it is a valid move. Otherwise google could cache it. – memo May 12 '12 at 12:59
  • That means Google should index the PDF (i.e. return it in the list of results) but not make the contents available from the Google cache? – Daan May 12 '12 at 17:01

2 Answers2

2

You can add robots directives to any file via the x-robots-tag http header. Setting it to noindex, follow sounds like what you want.

Tony McCreath
  • 2,882
  • 1
  • 14
  • 21
0

Im not sure but isnt this: <meta name="robots" content="noindex"> a good solution for your problem.?

memo
  • 441
  • 2
  • 6