I've built in a Logger (Monolog) in my PHP project that is configured to log system events (info/debug) but also errors (error/critical)
I've set it up so that when a critical error occurs 2 systems are called, one sends a mail and the other sends an sms.
Now in the event the database dies a critical event is logged and thus a mail and sms is sent. But since the site is probably going to attract some traffic this event will probably be triggered a few times a minute, at peak times even per second. In fact probably for each request to the site.
How can I prevent this from happening, prevent flooding/spamming myself with sms messages and mails? Should I alter my code with some sort of throttling that the same error message doesn't get sent within a given timespan (kinda hard to differentiate between error types), should I collect all errors and send them out once ever few minutes (gives slower response times) or are there systems that I can send my errors to that will take care of this for me? I'm thinking in the direction of a log collector that can be configured to send SMS/mails at certain severities but with some sort of throttling?