Stack Overflow
Stack Overflow

Questions tagged [spidermon]

Scrapy's spider monitoring library

Spidermon is Scrapinghub’s extension for monitoring Scrapy spiders. Spidermon makes it easy to validate data, monitor spider statistics and send notifications to everyone when things don't go well in an easy and extensible way.

7 questions
3
votes
1 answer

How to monitor specific spiders with Spidermon?

I'm following Spidermon's documentation on monitoring Spiders, in their examples, the monitors that they create seem to run across all of their Spiders, I can't figure out how to run a monitor on a single Spider. I've tried passing my Spider into…
paul
  • 31
  • 1
2
votes
0 answers

Is it possible to use a monitor on a script if it fails?

I use scrapinghub to run my spiders. I have a FinishReasonMonitor that slacks me if a spider fails. Is it possible to apply this to a script? My spiders rarely fail, but my scripts occasionally do. In scrapinghub it shows script outcomes as…
weston6142
  • 181
  • 14
1
vote
1 answer

Scrapy spidermon exceptions

I'm trying to setup the basic suite of spidermon monitors as described here I did a quick Google search and also found this. So I made a quick monitors.py, then copy and pasted the code in there. I then proceeded to do this: SPIDERMON_ENABLED =…
user3125823
  • 1,846
  • 2
  • 18
  • 46
1
vote
1 answer

Spidermon: How to include message in telegram bot?

I am trying to implement spidermon to monitor…
merlin
  • 2,717
  • 3
  • 29
  • 59
1
vote
1 answer

How to use Mandrill to send email report in Scrapy Spidermon

Currently, Scrapy Spidermon extension only shows an example to send email using Amazon Simple Email Service. Is it possible to use Mandrill instead? And how?
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
1
vote
2 answers

How to do Scrapy historical output comparison using Spidermon

So Scrapinghub is releasing a new feature for Scrapy quality insurance. It says it has historical comparison features where it can detect if the current scrape quantity is only below 50% of the previous scrape, which is suspicious. But, how can I…
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
0
votes
1 answer

Send Email Using Spidermon and Amazon SES

Hi i am newbie to spidermon, i have read the documentation but did not understand how I can send email with respective Body contents of error message. Can someone provide the code ? For your reference i would like to add the documentation as…
Karthi Thangavel
  • 5
  • 4