Scrape data every day on the same page and increment a Google Drive document

Question

I didn't manage to achieve a scheduled content scraping with Kimono Labs.

Here is what I intended to do: scrape a movie ranking which is published every day on a specific page and increment the data to an existing Google Sheet. In fact there would be 3 columns

Date / Ranking / Movie name

I achieved to create the scrape API with Kimono labs and to schedule it to run every day and the data looks good (beside I can't manage to add the date of the scraping as a a value).

But I can't make it increment new data on my Google Sheet, I can only update the existing data.

I investigated with Zapier of IFTTT but no luck either. Anyone has an idea of how I could manage this?

In what format does Kimono output data? Perhaps if you can give more detail about that, people can suggest how to write the data to a Google Sheet. Does it absolutely need to be a Google Sheet? You could use import.io and their API to render the data on your own website, would that work? — halfer, Aug 08 '15 at 12:21

score 0 · Answer 1 · answered Aug 09 '15 at 21:14

I'm trying to extract the following ranking. http://www.allocine.fr/film/attendus/

Kimono does the job I can extract the rank and the movie title. But this ranking changes everyday and I can't manage to keep the day of the ranking into a cell and write it into a google sheet document.

It is more understandable ?:)

score 0 · Answer 2 · answered Oct 29 '15 at 18:15

0

Probably You need some GAS scripting, for example Google App Script to insert data to a google spreadsheet via POST or GET

answered Oct 29 '15 at 18:15

AndriuZ

648
6
26

score 0 · Answer 3 · answered Oct 30 '15 at 00:05

Add this code to "Modify result" and you will have a date stamp :

function transform (data) {
   function add_date(item) {
        item.date = new Date();
        return item;
    }
     for (var collection in data.results) {
        data.results[collection] = data.results[collection].map(add_date);
    }


  return data;
}

Scrape data every day on the same page and increment a Google Drive document

3 Answers3