It sounds like the rule is cached somewhere/somehow.
From the AOT open the EventRule
table and verify the rule is not there.
Make sure your batch jobs are withhold/ended, then restart the AOS. It's reported (in URL below) that if the batch job is not in that state, even after an AOS restart, the alerts can keep coming.
Then restart your AOS(s) just in case the kernel caches the rule somehow. If you have multiple AOS's, just stop them all, then start them 1 by 1.
Read here for more info: https://daxteam.wordpress.com/2013/10/16/ax-2012-eventcud-constant-growth/
EventCUD
should hold a queue of alerts that need to be sent. If you are OK missing some alerts, just truncate the table.
Alternatively, running the job under [System Administration>Periodic>Alerts>Change based alerts] should read in all of those EventCUD
records and generate a ton of alerts that have yet to be sent out. If you choose this method, make sure to inform users they may receive a bunch of alerts that were pending.