5

The Google::Search module, which is based on the AJAX Search API, doesn't seems to work very well, or it is just me?

For example, I use firefox to search google for: http://bloggingheads.tv/forum/member.php?u=12129

It brings results.

But when I use the module this way:

$google_search = Google::Search->Web ( q => "http://bloggingheads.tv/forum/member.php?u=12129" );
@result =  $google_search->all;

I get nothing in the array.

Any idea?

Seems like this API doesn't bring the same results like when searching manually, am I missing something?

brian d foy
  • 129,424
  • 31
  • 207
  • 592
snoofkin
  • 8,725
  • 14
  • 49
  • 86
  • Just to clarify things, What I'm trying to achieve is the following: given a URL (like the one mentioned above...), check whether it has a cached version on google, if not , if it even appears on google. thats it. – snoofkin Sep 19 '10 at 00:38
  • You asked this question twice, so now people's advice for your single problem are spread over two separate threads. The other one is http://stackoverflow.com/questions/3744025/googlesearch-perl-module – brian d foy Sep 19 '10 at 14:13
  • I didnt, where? its a different question, and the URL you provided directs to here. – snoofkin Sep 19 '10 at 15:55

2 Answers2

1

Looking at the POD for Google::Search, it looks like it expects you to pass search terms to Web, instead of a URL. I downloaded a test script from CPAN, ran it, and it seems to produce expected results:

use strict;
use warnings;
use Google::Search;

my $search = Google::Search->Web(q => "rock");
my $result = $search->first;
while ($result) {
    print $result->number, " ", $result->uri, "\n";
    $result = $result->next;
}
print $search->error->reason, "\n" if $search->error;

__END__

0 http://www.rock.com/
1 http://en.wikipedia.org/wiki/Rock_music
2 http://en.wikipedia.org/wiki/Rock_(geology)
3 http://rockyourphone.com/
4 http://rockhall.com/
5 http://www.co.rock.mn.us/
6 http://www.co.rock.wi.us/
7 http://www.rockride.org/
etc...

I realize this does not specifically answer your question, but perhaps it steers you in the right direction.

toolic
  • 57,801
  • 17
  • 75
  • 117
  • Well, I even tried to pass: my $search = Google::Search->Web(q => "site: http://bloggingheads.tv/forum/member.php?u=12129" ); still I'm not getting the same results like when searching manually... it always fewer – snoofkin Sep 19 '10 at 00:42
1

I had a similar problem with cyrillic queries. Both Google::Search and REST::Google from CPAN didn't work for me - they were giving back fewer or no results compared to manual test.

Eventually I wrote a scraping module using WWW::Mechanize and HTML::TreeBuilder.

Here's a sample to get result stats:

my $tree = HTML::TreeBuilder->new_from_content($content);

if (my $div = $tree->look_down(_tag => 'div', id => 'resultStats')) {
    my $stats = $div->as_text();
}
else { warn "no stats" }
Eugene Yarmash
  • 142,882
  • 41
  • 325
  • 378