1

I'm processing M365 mailbox messages via MS Graph. I'm using .Net5 and the latest version of MSGraph SDK for .NET; (particularly the PageIterator for processing email messages) - but i'm actually experiencing the issue even via a pure call via Postman: in some cases the response is just truncated abnormally (hence the response JSON could not be parsed).

One example: ~56k messages are processed successfully, then during trying to get a next page by the iterator (for me seemengly randomly; some mailbox around 56k some at 78k, but almost always 50k+) i got a JSON parsing exception (sometimes unclosed string, sometimes unexpected char).

If i take the actual next page link from the iterator while catching the exception, i can reproduce the issue in Postman; the response is truncated.

In case i query the single message that is truncated separatelly via its id then the full message is available in the response.

An example call which fails has the response payload JSON truncated (but the call actually succeeds with HTTP 200):

https://graph.microsoft.com/v1.0/me/messages?$orderby=receivedDateTime+ASC&$select=ToRecipients,CcRecipients,Subject,From,Body,HasAttachments,ReceivedDateTime&$expand=attachments($select=name)&$top=32&$skip=57454

The end of the result json:

        "@odata.etag": "someetaghere",
        "id": "someiidhere",
        "receivedDateTime": "2017-03-19T09:15:42Z",
        "hasAttachments": false,
        "subject": "Fwd: Contrat Morval",
        "body": {
            "contentType": "text",
            "content":"Some text just an example which ends somewhere in the middle of the text

Some UPDATES for this particular case: In Postman

  • if i remove the "Body" param from the $select list of the above query, it constantly fails with "503 Service Unavailable" after long (<~20sec) response times
  • unless if i set the $top param to 31 or lower, then everything works OK, regardless if "Body" is iuncluded in the $Select list or not
  • if i use $top>31 with "Body" inlcuded, the response payload is truncated always at the same position of the 24th item in the result array, regardless of the value of $top

I hoped if i use 30 as page size running my Graph SDK code then i could forget this bug :), but unfortunatelly there i receive "503 Service Unavailable" for the same query that succeeds in Postman, with message

Code: generalException Message: Unexpected exception returned from the service. ClientRequestId: 610103aa-ac07-4b8b-b7af-0aa7bdbcce0e

The Timestamp form the response headers: Thu, 03 Dec 2020 10:49:36 GMT

Any help would be appreciated, how could i ensure that the message is loaded correctly? I tought about some throttling, quota/message size limit or restriction, but i could not find anything - and now I can reproduce the issue in postman anytime.

Thanks

rpart
  • 41
  • 4
  • (1) When you say the API call fails, does it mean you get the error response back. If yes, share the requestid, timestamp for it – Dev Dec 03 '20 at 07:59
  • (2) I see that you're using mix of odata filters. I would see if you can start by adding one by one and see at which point you can see the issue. This will help you to isolate the issue outside of your code as well. – Dev Dec 03 '20 at 07:59
  • Hi @Dev, (1) thanks for the tips, i have updated my post with details. (2) I played with params as detailed in the post, but i do not see a consistent problem, but problems when having a mix of conditions with $top and $select usage toegether.. i'm pretty cluless frankly :| – rpart Dec 03 '20 at 11:01
  • Yes i guessed so. Thanks for the detailed info/updates. I think you have the answer for the above now. When you process huge amount of data from mailbox, then you need to be careful. Before i proceed with production mailboxes i would have tested internally, make sure all my test cases passes it - at least it will work as expected. Prod environments always be different. You need to keep the $top/page size and its limit (say, 30). I would not say it's bug, rather it may be by design or a limit set for the API. – Dev Dec 03 '20 at 17:47
  • Thanks, but why do you think i have the answer? I still do not know the reason for the errors nor i have any solution. Btw this came up during my testing indeed, but regardless the MS Graph API is in production; and based on the docs the API should work with any $top/page size between 1 and 999. The strange thing it is works OK with most of the tested mailboxes, but some bigger/older ones failing as explaned. I dont think it is a limit issue, as i can get literally 10 times bigger responses both in size and lenght then the problematic query, and those are not truncated. – rpart Dec 03 '20 at 21:47
  • When i said "you have the answer", its just based on that way you troubleshooted and found it works for certain page size. Being said that if the given API is designed to work in page size and if its in v1.0(not in beta), then it will work. I would suggest you to check with Microsoft support they may assist you on this. – Dev Dec 04 '20 at 13:43
  • I get the same issue when using PowerShell to query Azure AD Audit Logs. There is a property called resultReason. In Graph Explorer it's a very long message. In the Invoke-RestMethod response, it's half a line. – NadimAJ Nov 12 '21 at 17:25

0 Answers0