0

I'm building a chat app. Each channel has many messages. I am building the channel list view where I want to display all the channels sorted by the most recent message sent at per channel.

Each time a message is sent or received I would like to keep the channel.latestMessageUpdatedAt up to date so that I can sort channels later.

I would like to separate concerns and not have to remember to update channels each time messages are updated.

My strategy is to update the channel inside the listener to the message realm, but I get the following error

Error: Wrong transactional state (no active transaction, wrong type of transaction, or transaction already in progress)
const ChannelSchema = {
  name: "channel",
  primaryKey: "id",
  properties: {
    id: "string",
    latestMessageUpdatedAt: "date",
  },
};

const MessageSchema = {
  name: "message",
  primaryKey: "id",
  properties: {
    id: "string",
    channelId: "string",
    text: "string",
    updatedAt: "date",
  },
};

const realm = await Realm.open({
  path: "default.realm",
  schema: [ChannelSchema, MessageSchema],
  schemaVersion: 0,
});

const messagesRealm = realm.objects("message");
messagesRealm.addListener((messages, changes) => {
  for (const index of changes.insertions) {
    const channel = realm.objectForPrimaryKey(
      "channel",
      messages[index].channelId
    );
    if (!channel) continue;

    const message = messages[index];
    channel.latestMessageUpdatedAt = new Date();
  }
});

I've checked the docs there seems to be no reason why this wouldn't be possible.

Perhaps there is a better way of having this computed field.

Note I thought about having embedded objects / a list of messages on the channel but the number of messages could be up to 10k, I don't want that all returned at once into memory.

I've also tried doing

realm.write(() => {
    channel.latestMessageUpdatedAt = new Date();
});

but I get the error that a transaction is already in progress.

david_adler
  • 9,690
  • 6
  • 57
  • 97
  • Hm. The question is a bit unclear. Do you want to add a listener to a Realm? Or do you want to add a Listener to a specific object *within* a realm? Also, your code doesn't reference *channels* anywhere - did you mean *channel*? – Jay Feb 01 '22 at 21:21
  • I already have a listener on the messages realm, I want to update the channels realm from inside the messages realm listener – david_adler Feb 01 '22 at 22:48
  • There's only one `realm` in your code, not a messages realm and a channels realm. Your question mentions a - *channels collection* - but your code shows `channel` is that a typo? Also, you cannot modify a managed object outside of a write transaction which is what it appears your code does `channel.latestMessageUpdatedAt = new Date();` – Jay Feb 02 '22 at 00:14
  • Hey @Jay I've updated the question to have a more complete code snippet – david_adler Feb 02 '22 at 09:24
  • Just a few thoughts. Naming vars clearly in code is important. It does not affect the operation but this `const messagesRealm` is not accurate, as Realm can have different Realm 'files' on disk and that's not what that var represents. A better name would be `const allMessages` - because that's what the call does; gets all messages. It's a personal choice though. The other thing is there's really no relationship between channels and messages; why don't you create a List in channels that forward links to messages and then a inverse relationship between messages back to channels? That would... – Jay Feb 02 '22 at 18:25
  • ... make it much easier to keep everything organized and, since Lists are always kept in order, the last message in that last would always be the last message added, making sorting channels a snap. – Jay Feb 02 '22 at 18:25
  • Firstly, I don't think I can sort the channels by the first item of the list? – david_adler Feb 02 '22 at 18:33
  • Secondly, the channel could have hundreds of thousands of messages. I don't want to pull that entire list into memory when querying for the most recently updated channels – david_adler Feb 02 '22 at 18:35
  • RE naming, open to naming suggestions and I think my naming isn't great. I think `messagesCollection` would be the best. – david_adler Feb 02 '22 at 18:38
  • *sort the channels by the first item of the list* - you won't need to to; sort the `messages` by timestamp to get the LAST message and then get the channel from that, or the last three messages, linked channels. The LAST message is the one most recently updated. *channel could have hundreds of thousands of messages* - yep, the beauty of Realm. Objects are lazily loaded so large lists of data will have very little memory impact. – Jay Feb 02 '22 at 20:09
  • `sort the messages by timestamp to get the LAST message and then get the channel from that, or the last three messages, linked channels.` No this will return the three most recent messages across all messages. I want to return the channels sorted by the most recent message. If you have an alternate solution please post an answer. Thanks! – david_adler Feb 02 '22 at 21:27
  • The last message (the most recent) will be linked to and associated channel. If you take the second to the last message, you can then get that associated channel - they will be in the correct order. If you don't want a channel listed twice (which could happen if the last two messages are in the same channel), use `distinct` in code after you get the channels. e.g. once you have the last, say 5 channels, you can use distinct to remove the dups. I can't post an answer as this is a Java question and we are all Swift based. I crafted a small app to do what I suggest, which works, so it's doable. – Jay Feb 02 '22 at 23:03
  • this is a javascript question. Is distinct a realm query or do i have to do it in memory? – david_adler Feb 02 '22 at 23:18
  • Well, yes. There are a number of options. It really depends on what your UI does and depends on the number of channels you may have. Again, as long as you use Realm objects they are all lazily loaded so memory isn't an issue. If you cast them to an array for an example it occupies memory but a couple hundred objects is no big deal. Another thought occurred to me and just thinking out loud; have a function in your Channel object to where you could pass it a new message to be added to it's `messages` List object, and it would update the channels `latestMessageUpdatedAt` property. – Jay Feb 02 '22 at 23:25
  • I think the issue with that approach is that I want to return all the channels. The oldest channel could have had its latest message sent 10k messages ago. I would have to iterate through all the messages ever sent to get all the channels. Think of WhatsApp chat list view – david_adler Feb 02 '22 at 23:40
  • Also feel free to write an answer / link to some code in swift. I've done some swift and the API should be pretty translatable. – david_adler Feb 03 '22 at 09:39
  • Also @Jay `Realms are the core data structure used to organize data in Realm Database. At its core, a realm is a collection of the objects that you use in your application, called Realm objects, as well as additional metadata that describe the objects.` https://docs.mongodb.com/realm/sdk/react-native/fundamentals/realms/ maybe `messagesRealm` wasn't such a bad name – david_adler Feb 03 '22 at 14:31
  • Thanks for letting me know what Realm is.. only been using it for 8+ years now! lol. j/k. You pick a naming convention that works for you; in one of our apps we have inventory stored in a Realm called InventoryRealm. Another Realm stores customers; CustomersRealm. Those are separate discreet Realms (local files stored on the drive), so in code when we are working with the Inventory, we open it and our var is called `inventoryRealm`. In your case `messagesRealm` is not realm - it a collection of messages objects. Again though, name it what works for you. – Jay Feb 03 '22 at 16:58
  • *I want to return all the channels* - that may not be the best user experience; what if there are 10,000 channels? Paginating data is best practice; only display what the user is interested in or what 'fits' in the UI. If iPhone, you probably only want to show 10-15 at a time. *iterate through all the messages* - no iteration needed. If you go with my second suggestion above, you would only need to pick off the last 10-15 channels (if they are sorted ascending by timestamp). Otherwise, you would get the last 10-15 messages (assuming they are sorted) and get the channels via the backlink. – Jay Feb 03 '22 at 17:05
  • Yes that's fine but worst case I will have to iterate through all the messages. And I will need to resort the list every time a message is sent. For example suppose there are only 2 channels the first channel has 100k messages. The second channel has only one message but it is older than all 100k messages from the first channel. Do you see where I am going? – david_adler Feb 03 '22 at 18:43

1 Answers1

1

The OP requested a Swift solution so I have two: which one is used depends on the data set and coding preference for relationships. There is no automatic inverse relationship in the question but wanted to include that just in case.

1 - Without LinkingObjects: a manual inverse relationship

Let's set up the models with a 1-Many relationship from a channel to messages and then a single inverse relationship from a message back to it's parent channel

class ChannelClass: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    @Persisted var channelName = ""
    @Persisted var messageList: List<MessageClass>
}

class MessageClass: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    @Persisted var msg = ""
    @Persisted var msgDate = ""
    @Persisted var myChannel: ChannelClass!
}

Then after we populate Realm we have some objects that look like this - keeping in mind that different channels will have had messages added at different times

channel 0
   message 1
   message 3
   message 4

channel 1
   message 5
   message 9
   message 10

channel 2
   message 2
   message 6
   message 7
   message 8

It would look like this because: suppose a user posts a message to channel 0, which would be message 1. Then a day later another user posts a message to to channel 2, which would be message 2. Then, on another day, a user posts a message to channel 0 which would be message 3. Etc etc

Keeping in mind that while Realm objects are unsorted, List objects always maintain their order. So the last element in each channels list is the most current message in that channel.

From there getting the channels sorted by their most recent message is a one liner

let messages = realm.objects(MessageClass.self).sorted(byKeyPath: "msgDate").distinct(by: ["myChannel._id"])

If you now iterate over messages to print the channels, here's the output. Note: This is ONLY for showing the data has already been retrieved from Realm and would not be needed in an app.

Channel 0 //the first message
Channel 2 //the second message
Channel 1 //the third message? Not quite

Then you say to yourself "self, wait a sec - the third message was in channel 0! So why output channel 1 as the last item?"

The reasoning is that the OP has a requirement that channels should only be listed once - therefore since channel 0 was already listed, the remaining channel is channel 1.

2 - With LinkingObjects: Automatic inverse relationship

Take a scenario where LinkingObjects are used to automatically create the backlink from the messages object back to the channel e.g. reverse transversing the object graph

class MessageClass: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    
    @Persisted var msg = ""
    @Persisted var msgDate = ""
    @Persisted(originProperty: "messageList") var linkedChannels: LinkingObjects<ChannelClass>
}

the thought process is similar but we have to lean on Swift a little to provide a sort. Here's the one liner

let channels = realm.objects(ChannelClass.self).sorted { $0.messageList.last!.msgDate < $1.messageList.last!.msgDate }

What were doing here is querying the channels and using the msgDate property from the last message object in each channels list to sort the channels. and the output is the same

Channel 0 //the first message
Channel 2 //the second message
Channel 1 //see above

The only downside here is this solution will have larger memory impact but adds the convenience of automatic reverse relationships through LinkingObjects

3 Another option

Another options to add a small function and a property to the Channel class that both adds a message to the channels messagesList and also populates the 'lastMsgDate' property of the channel. Then sorting the channels is a snap. So it would look like this

class ChannelClass: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    
    @Persisted var channelName = ""
    @Persisted var messageList: List<MessageClass>
    @Persisted var lastMsgDate: String
    
    func addMessage(msg: MessageClass) {
        self.messageList.append(msg)
        self.lastMsgDate = msg.msgDate
    }
}

When ever a message is added to the channel, the last message date is updated. Then sort channels by lastMsgDate

someChannel.addMessage(msg: someMessage)

Note: I used Strings for message dates for simplicity. If you want to do that ensure it's a yyyymmddhhmmss format, or just use an actual Date property.

Jay
  • 34,438
  • 18
  • 52
  • 81
  • As you have noted, the first two approaches, worst case will have to iterate through everything so don't really work for me. In option 1, worst case, iterate through every message. Option 2, every case, iterate through every channel. The final approach seems solid and is what I've settled for but unfortunately I do have to remember to always call `addMessage`. I was hoping for a more expressive way of doing that. – david_adler Feb 04 '22 at 09:06
  • @david_adler I am sorry, but I have no idea what you mean by *iterate*. There is no iteration in the answer (to get the data) what-so-ever - *and it's not needed!*. The only iteration I mentioned was to simply **output the results to the console**. The code provided in the answer works and does exactly what you asked; it's pretty much a copy/paste from a working project (which is why I provided the output from console). Perhaps if you can explain why you need to iterate over something, maybe I could refine the answer? – Jay Feb 04 '22 at 16:37
  • `realm.objects(MessageClass.self).sorted(byKeyPath: "msgDate").distinct(by: ["myChannel._id"])` ah sorry, I thought [`distinct`](https://docs.mongodb.com/realm-sdks/swift/latest/Protocols/RealmCollection.html#/s:10RealmSwift0A10CollectionP8distinct2byAA7ResultsVy7ElementQzGqd___tSTRd__SSAHRtd__lF) here was actually a swift function which would have meant it may have to go through all messages in memory to find the oldest channel. I think [`distinct` is not present in the javascript API](https://docs.mongodb.com/realm-sdks/js/latest/Realm.Collection.html). – david_adler Feb 04 '22 at 20:15
  • That said, under the hood won't realm have to do something like `for message in sortedMessages: if isDistinct(message.id) yield message` which still worst case may require realm to iterate over every message? Worst case being the oldest channel's latest message is the oldest message. – david_adler Feb 04 '22 at 20:17
  • (For the above two comments I'm talking about the option 1 you have proposed. Thanks for your time! – david_adler Feb 04 '22 at 20:17
  • I am not a javascript guy but even if `distinct` is not available you can probably do the same thing with a server side function or possibly a REST call. Oh - `distinct` is available as a [MongoDB Action](https://docs.mongodb.com/realm/mongodb/actions/#std-label-mongodb-actions). And to answer the question about under the hood; MongoDB has its own way of storing data - so while you think of it as iterating, that may not be the case at all. Bottom line there is not to focus on that aspect as tens of thousands of records can be processed in milliseconds. – Jay Feb 04 '22 at 21:10
  • I'm not using mongodb btw. Only realm locally. – david_adler Feb 04 '22 at 22:26
  • Hmm. How are you going to have a chat app - *I'm building a chat app* - when all of the data is stored locally? Wouldn't a chat app require that multiple users can chat back and forth; therefore MongoDB Realm cloud storage would be a requirement? – Jay Feb 04 '22 at 22:31
  • Cloud storage is not really required for a chat app. Take sms or WhatsApp. All messages are stored on a device database and the server is mostly just a relay. – david_adler Feb 05 '22 at 12:17
  • But anyway, that’s not my case. All messages are stored on a remote server but also stored on device, it’s just that my backend db is not mongo, is all I’m saying. So how mongo indexes data is not really relevant to realm. – david_adler Feb 05 '22 at 12:19
  • Also my bet would be distinct queries will still do a full table scan in mongo as with most databases. – david_adler Feb 05 '22 at 12:24
  • "server is mostly just a relay" that's cloud storage as the server also stores that data; whatsapp definitely leverages *both* cloud storage and device storage as that's why you can chat via a web app as well as a local device. See whatapp web. Realm is an end-to-end product and can handle user auth, local storage and sync'd cloud storage so it's an ideal single platform for a chat-type app; it's scalable and easily maintainable, instead of two totally different databases. I suggest you give it a try - I think you'll be surprised at the performance and how quickly your app comes together. – Jay Feb 05 '22 at 16:27
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/241758/discussion-between-david-adler-and-jay). – david_adler Feb 05 '22 at 18:42