What you're describing is functionality that's more closely tied to periodic background sync, rather than push notifications, in that push notifications require a network connection. Periodic background sync has the advantage of being network-independent. (Periodic background sync is just a proposal right now, and hasn't been implemented in any browsers.)
But, unfortunately for your use case, periodic background sync is explicitly not designed for events that need to be triggered at a precise time, which would be needed to make an alarm clock useful. As per the original proposal:
What periodic sync is not
Periodic sync is specifically not an exact alarm API. The scheduling
granularity is in milliseconds but events may be delayed from firing
for several hours depending on usage frequency and device state
(battery, connection, location).
The results of a sync running should be "beneficial" not "critical".
If your use-case is critical, one-off syncs or the push API may serve
your requirements.
There are various optimizations baked into modern mobile operating systems to minimize power usage when a device is considered idle, including running periodic tasks in staggered batches instead of given those tasks fine-grained control over when they're run.