0

I need xml file for indexing my website for google crawling. I'm using some software to make XML file. My question is do I need to list all dynamic pages. I mean like this:

http://mysite.com/page/?id=01
http://mysite.com/page/?id=02
http://mysite.com/page/?id=03
http://mysite.com/page/?id=04
http://mysite.com/page/?id=05

if yes, why is that? and what is going to happend if I wouldnt include them and just say:

http://mysite.com/page/

If I include all the id's the result would be a huge XML file. Does google accept this such a large file or they have limit for it?

Thanks in advance for all help and time.

John Conde
  • 217,595
  • 99
  • 455
  • 496
Jay
  • 1,384
  • 1
  • 17
  • 30

3 Answers3

1

Google isn't going to index all your dynamic pages anyways. It will throw many of them out even if you put them in the sitemap.xml. The content will be too similar.

There is a limit to the number of entries in a sitemap.xml It used to be ~50k pages/10MB. In my experience Google will crawl a few thousand and stop if they look too similar and have no inbound links.

Byron Whitlock
  • 52,691
  • 28
  • 123
  • 168
0

You do not need an XML sitemap at all. It just makes it easier for google to crawl your content. And obviously you don't have to put dynamic stuff in it.

ThiefMaster
  • 310,957
  • 84
  • 592
  • 636
  • You mean if I make XML sitemap for my site, I make crawling harder for google? – Jay Jan 24 '11 at 22:47
  • No. A sitemap makes it easier because it gets all the URLs you consider important at once. But it will most likely still check for links on your page. – ThiefMaster Jan 24 '11 at 22:50
0

If this is a real issue, try reading up on rel="canonical" which is made to exclude those types of pages from Google. While it's usefulness is based on use case, you may find it is the right solution for you.

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=139394

fiiv
  • 1,267
  • 1
  • 8
  • 16