0

I have built a Twitter client that's only one page: index.html. When being used, it draws in content from JSON feeds to populate various columns. The issue is hardly any traffic is coming in from Google except for searches on the site's domain name. This is because Google only crawls index.html when it's blank and before the user enters any Twitter username to be looked up.

How can I have the Google spider, when attempting to crawl index.html crawl instead of just index.html (or in addition) also say index.html#CNN (ie site.com/#CNN) or index.html#whitehouse whereby all content from these pages will also be crawled and thus be accessible to users searching on the terms therein.

Frank Farmer
  • 38,246
  • 12
  • 71
  • 89
Adrian33
  • 281
  • 5
  • 16
  • In case you don't already know, google bot does not run javascript when it comes to your page. – Tony Jun 24 '09 at 19:27
  • Google doesn't index 100% AJAX sites well. If you want to rely on google for traffic, you're going to have to bite the bullet and settle on a slightly less AJAX-centric design. – Frank Farmer Jul 10 '09 at 22:24

2 Answers2

2

I don't think Google will ever include an fragment (#) in its index. You should provide an alternate URL, like site.com/CNN

Greg
  • 316,276
  • 54
  • 369
  • 333
  • Problem here is that I don't want such a page being named as such, as it would not reflect the page's content as soon as the user moves away from CNN. But, it may be the only choice. – Adrian33 Jun 24 '09 at 19:21
0
  • Use a site map to illustrate the options for Google.
  • Embed hidden links so that Google knows to look there?
  • Use a separate page to list some "examples". Could be a hybrid for your users plus Google...
Frank V
  • 25,141
  • 34
  • 106
  • 144
  • the site map might work, but there is already one link on the home page to site.com/#testuser (some "examples" exist, but only in a div that draws content from a text file when loaded in a browser (not crawler?)) Content from site.com/#testuser is not crawled. – Adrian33 Jun 24 '09 at 19:19