correlating messages through moves via EWS

Question

Let's say you are building an application that syncs message bodies and metadata down from EWS. Getting your initial state synchronized is fairly straightforward: call SyncFolderHierarchy to get the folder list, SyncFolderItems to get the message ItemIds, and GetItem to get the message bodies and metadata.

Things become a bit more complicated when trying to track messages through moves. After a move, calls to SyncFolderItems would return a Create in one folder and a Delete in another. You'd like to correlate these so that the client could avoid redownloading the message body and attachments. (Also, so that the client doesn't lose any metadata it had associated with its local copy.) However, moving a message between folders changes its EWS ItemId, so the ItemId can't be used to correlate the Create and the Delete.

The EWS documentation suggests subscribing to streaming notifications, which do indeed support Move events. But streaming notifications aren't buffered when the stream is not connected, so you still have to bring the client back in sync before establishing a streaming connection. So streaming notifications can't be a full correlate-through-moves solution.

Another EWS option is subscribing to pull notifications. Like streaming notifications, pull notifications support Move events. And unlike streaming notifications, pull subscriptions buffer changes. But if your client is offline when the pull subscription expires, you are back in the same situation. (Still, since a pull subscription can be scoped to last a full day, this may still be workable.)

The last option is using something other than the ItemId to correlate moved items through SyncFolderItems:

• PR_SEARCH_KEY works through moves but has problems with copies (as the copy ends up with the same PR_SEARCH_KEY as the original)

• PR_ENTRYID seems like it'd be workable for this purpose and seems better than PR_RECORD_KEY:

A MAPI store provider assigns a unique ID string when an item is created in its store. Therefore, the EntryID property is not set for a Microsoft Outlook item until it is saved or sent. The EntryID changes when an item is moved into another store, for example, from your Inbox to a Microsoft Exchange Server public folder, or from one Personal Folders (.pst) file to another .pst file. Solutions should not depend on the EntryID property to be unique unless items will not be moved. The EntryID property returns a MAPI long-term EntryID.

So... What's the right way to correlate items through moves via EWS?

Answer 1

There are two things that may help you:

1st approach

We solved this by giving each item (calendar items in our case) a User Defined property (a.k.a. Extended Property) containing a copy of the EWS Item ID (whenever we write/update it).

Whenever we read the item for synchronisation, we check if that property still matches the EWS Item ID. If not, we know that the item has been moved and handle it accordingly (and we clear the User Defined property).

Together with some other User Defined properties for bookkeeping, this lets us maintain the integrity between the items in Exchange and the ones in our application.

2nd approach

There are some edge cases where this does not work (but we ignore them). For those, using the EWS Item ID for tracking no longer works, and it is recommended to use the UID/PidLidGlobalObject properties. I did not have to actually implement this, but here are some references to get you started:

"Exchange calendar: Is ConversationId a good identifier of master events for FindItem occurrences?"
"The appointment.Id.UniqueID changed"
"EWS API- Differences in ICalUid returned when appointments are created by Office 365 account vs. Microsoft Outlook Mac Client"
"EWS: UID not always the same for orphaned instances of the same meeting"

"Property: UID"
"PidLidGlobalObjectId Canonical Property"
"PidLidCleanGlobalObjectId Canonical Property"
"Getting GlobalObjectID from UID(Appointment created from IPhone)"
"Developer information about the calendar changes in Outlook 2003 Service Pack 2, in Exchange Server 2003 Service Pack 2, and in later versions of Exchange Server and of Outlook"

"Working with extended properties by using the EWS Managed API 2.0"

I cannot give you specifics on what code to use, because our app is written in Delphi and uses SOAP calls for synchronizing calendar items only.

Answer 2

Another option you have is the Message ID field (from RFC 5322), i.e. internetMessageId from the Graph API message resource. That ID remains constant across moves. However, it remains constant in copies too, which may or may not be what you like.