5

I am running into an issue with NotificationHubs where occasionally notifications silently fail to get delivered to an iOS client.

My Notification Hub is setup to use token authentication with APNS (as opposed to the legacy certificate authentication).

I updated my notification hub pricing tier to standard so I could get some more information about it. Most of the time (over 95%) notifications go through correctly. I added logging to track the NotificationId of each push notification that was queued with Notification Hubs. Then, when I was alerted of a failure, I went and looked up the details for that specific notification via the following method:

var details = await notificationHub.GetNotificationOutcomeDetailsAsync("<notification id>");

Inspecting the details I noticed that while the State was "Completed" (meaning NotificationHubs had received and processed the operation) the PnsErrorDetailsUri had a non-null value, indicating there was an issue delivering the notification:

enter image description here

Navigating to the value of the PnsErrorDetailsUri in a browser caused the following file to be downloaded:

enter image description here

In here, I noticed that the NotificationSystemError says there was an "InvalidToken". This token seems like it should be related to some "under the covers" communication between Azure and APNS. It is definitely NOT due to the device token registered in NotificationHubs being invalid. I verified that the registrationId was still in notification hubs, and that it pointed to the correct device. In addition, grabbing the raw NotificationBody from the details and re-submitting it with the same tag causes the new notification to be delivered successfully.

Does anyone know what the InvalidToken may be referring to, or what could be the cause of these intermittent NotificationHub delivery failures?

UPDATE:

I have found mention of the different NotificationSystemErrors here, one of them being my InvalidToken error. However, I can't find a description of what the actual causes of these errors are.

lehn0058
  • 19,977
  • 15
  • 69
  • 109
  • Is authentication enabled on your service? If yes, you will need to present an X-ZUMO-AUTH header in the request that contains a valid access token. Otherwise, it will cause a invalid token error. – Amor Jul 13 '17 at 09:18
  • I am using the C# NotificationHubs nuget package, which should be managing all of the headers for me. Also, my notifications are being queued from an azure function that is triggered by an Azure Service bus queue - no authentication features are enabled. – lehn0058 Jul 13 '17 at 11:09
  • Are you still experiencing this issue ? Could you share us the namespace & hub details, along with timeframe to nhsloop@microsoft.com to investigate this further ? – Sateesh Jul 18 '17 at 06:09
  • Yes, we are still experiencing the issue. I have the information to send you requested, but unfortunately nhsloop@microsoft.com bounces my email back with the error: “The group nhsloop only accepts messages from people in its organization or on its allowed senders list” – lehn0058 Jul 18 '17 at 11:22
  • Could you try sending details to nhubsupport@microsoft.com. – Sateesh Jul 20 '17 at 16:26
  • Thanks, email sent. – lehn0058 Jul 20 '17 at 16:29

1 Answers1

0

I never really got a definitive answer why the error was happening, but I appear to have been able to resolve my issue.

We have 2 separate notification namespace/hub, one for apple production notifications and one for the apple sandbox notifications. We have a switch in place so devices register with the correct hub. I investigated all of our registrations, and they all looked to be in the correct location.

However, during this inspection I noticed that many devices had a large number of registrations. Each of these registrations had the SAME apple PNS identifier (which was a valid token), but it seemed odd to me that there were dozens (in one case hundreds) of the same PNS token registered. Each RegistrationID was the same, except it has a hyphenated incremented number after it (for instance, 1231231231235396312-6910179870480973035-1, 1231231231235396312-6910179870480973035-2, 1231231231235396312-6910179870480973035-3, etc.). It looks like each time I call NotificationHubClient.CreateAppleNativeRegistrationAsync, it is adding a new entry without de-duping. Clearing out these duplicate items seems to have resolved the issue I was encountering. Seems like NotificationHubs was getting confused sometimes with too many registrations being linked to a device.

I ended up adding some code on my end to attempt to filter out duplicates for the time being. However, I would expect that NotificationHubs should be handling this for me...

lehn0058
  • 19,977
  • 15
  • 69
  • 109