2

I a wrote recipe screen scraper. Because of how easy it is for screen scrapers to break I wrote some specs that test various websites for the expected output. On my development enviorement i'm using VCR so I don't have to keep making calls to those sites.

The problem is that doesn't really tell me if the recipe sites change and break my scraper . I could periodically refresh the cassettes, which I do, but I'd like to be more proactive. I'd like someway to run these specific specs daily and get notified of the outcome if they fail so I can go back and fix the scraper .

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
hadees
  • 1,754
  • 2
  • 25
  • 36

1 Answers1

3

One approach is to package your app as a command line tool that you can then schedule as a cron job. When things fail, you can have it email you or use some notification library on your dev machine to get your attention. I've done similar things using Growl.

If you don't want to rely on your dev box, make an app you can host on Heroku that does the same thing (emails you upon failure).

Hope that helps.

jboursiquot
  • 1,261
  • 11
  • 8
  • I did think about pull the testing code out of a spec and just turning it into a library because all i'm really doing is making sure a hash I get back from my scraper equals a known hash of values but that still seems silly to me. – hadees Nov 30 '12 at 01:33
  • It doesn't have to be a "library" per se, just a small app with a very specific purpose that you can fire up from the command line. Nothing wrong with that :) – jboursiquot Nov 30 '12 at 01:38