Download the posts from a Google group before resetting

Question

I want to reset one of my groups (a class discussion), but I would like to retain the discussion for reference. There aren't many posts (maybe 50), and I could do it by hand, but is there a way to do that through google apps scripts or python?

I found a few possibilities, but neither in a language I'm familiar with (though I might be able to translate):

this link: http://saturnboy.com/2010/03/scraping-google-groups/

this Perl code:

#!/usr/bin/perl
# groups2csv.pl
# Google Groups results exported to CSV suitable for import into Excel.
# Usage: perl groups2csv.pl < groups.html > groups.csv

# The CSV Header.
print qq{"title","url","group","date","author","number of articles"\n};

# The base URL for Google Groups.
my $url = "http://groups.google.com";

# Rake in those results.
my($results) = (join '', <>);

# Perform a regular expression match to glean individual results.
while ( $results =~ m!<a href=(/groups[^\>]+?rnum=[0-9]+)>(.+?)</a>.*?
<br>(.+?)<br>.*?<a href="?/groups.+?class=a>(.+?)</a> - (.+?) by 
(.+?)\s+.*?\(([0-9]+) article!mgis ) {
    my($path, $title, $snippet, $group, $date, $author, $articles) =
        ($1||'',$2||'',$3||'',$4||'',$5||'',$6||'',$7||'');
    $title =~ s!"!""!g; # double escape " marks
    $title =~ s!<.+?>!!g; # drop all HTML tags
    print qq{"$title","$url$path","$group","$date","$author","$articles"\n\n};
}

Answer 1

Take a look at the HTTrack utility mentioned in this webapps question and in this forum discussion.

Note I'm assuming you don't actually want to screen scrape and process data but merely have a copy of the discussion for future reference.

EDIT: If you actually want to screen scrape, you can do this too but writing a script to do it can be a significant time sink. Screen scraping is more about extracting specific pieces of data from an html document than it is grabbing the entire html document. An example where you might need to screen scrape would be if you were looking at the jeopardy website and wanted to grab individual questions, their point values, who answered them right, which game they occurred in, etc for insertion into a database.