ReactiveX: Group and Buffer only last item in each group

Question

How to group an Observable, and from each GroupedObservable keep in memory only the last emitted item? So that each group would behave just like BehaviorSubject.

Something like this:

{user: 1, msg: "Anyone here?"}
{user: 2, msg: "Hi"}
{user: 2, msg: "How are you?"}
{user: 1, msg: "Hello"}
{user: 1, msg: "Good"}

So in memory we'd have only have the last item for each user:

{user: 2, msg: "How are you?"}
{user: 1, msg: "Good"}

And when a subscriber subscribes, these two items were issued right away (each in it's own emission). Like we had BehaviorSubject for each user.

onCompleted() is never expected to fire, as people may chat forever.

I don't know in advance what user values there can be.

Its unclear from your description what format you want the results in. Do you want a single stream that emits an array of the last items? Or do you want separate `Observables` for each user? — paulpdaniels, Dec 08 '15 at 02:00
@paulpdaniels Good question. The answer is, well, doesn't really matter. My task needs single stream, but if I have multiple `Observables`, then I'll just `merge` them into one. It's better to get each item in it's own emission, although an array of items in one emission would work too. — Dzmitry Lazerka, Dec 08 '15 at 03:30

score 3 · Accepted Answer · answered Dec 08 '15 at 18:32

3

I assume your chatlog observable is hot. The groupObservables emitted by #groupBy will consequently also be hot and won't keep anything in memory by themselves.

To get the behavior you want (discard everything but the last value from before subscription and continue from there) you could use a ReplaySubject(1).

Please correct me if I'm wrong

see jsbin

var groups = chatlog
      .groupBy(message => message.user)
      .map(groupObservable => {
        var subject = new Rx.ReplaySubject(1);
        groupObservable.subscribe(value => subject.onNext(value));
        return subject;
      });

answered Dec 08 '15 at 18:32

Niklas Fasching

1,326
11
15

Yes, chatlog in example is hot. Thanks, that was my first thought. But the problem is that this solution kicks in only when someone subscribes to the `groups` observable. In your jsbin example, try to put `groups.subscribe(...)` inside a `setTimeout()` -- then first messages that were emitted before `groups.subscribe()` is called are lost forever. – Dzmitry Lazerka Dec 10 '15 at 21:09
You can fix it by making `var log = new Rx.ReplaySubject()` -- but then we'd keep all those messages in memory. So I don't know how to fix it. The only way I see is to develop my own Subject, with it's own State. Just like BehaviorSubject, but the State would hold a Set, not just one value. – Dzmitry Lazerka Dec 10 '15 at 21:11
Sorry, just needed to call dummy `.subscribe()` without parameters before issuing any `onNext()`. – Dzmitry Lazerka Dec 11 '15 at 10:28
Note that you have to subscribe to the subjects emitted by `groups` straight away when you subscribe to `groups` otherwise there is a risk that you might loose some messages (subjects emitting values before you subscribe to them - subjects do not need to be kicked in). – user3743222 Dec 11 '15 at 11:53
Could you elaborate on what did the trick in the end? From my understanding you subscribe to the GroupedObservable as soon as possible and hold on to the GroupObservables it emits without subscribing to them. Subscription to the captured GroupObservables happens only at the moment you want to drop into the chat. Is that correct? – Niklas Fasching Dec 12 '15 at 00:01

user3743222 · Answer 2 · 2015-12-11T01:29:45.677

2

You can write the reducing function that turns out the latest emitted items of grouped observables, pass that to a scan observable, and use shareReplay to recall the last values emitted for new subscribers. It would be something like this :

var fn_scan = function ( aMessages, message ) {
  // aMessages is the latest array of messages
  // this function will update aMessages to reflect the arrival of the new message
  var aUsers = aMessages.map(function ( x ) {return x.user;});
  var index = aUsers.indexOf(message.user);
  if (index > -1) {
    // remove previous message from that user...
    aMessages.splice(index, 1);
  }
  // ...and push the latest message
  aMessages.push(message);
  return aMessages;
};
var groupedLatestMessages$ = messages$
    .scan(fn_scan, [])
    .shareReplay(1);

So what you get anytime you subscribe is an array whose size at any moment will be the number of users who emitted messages, and whose content will be the messages emitted by the users ordered by time of emission.

Anytime there is a subscription the latest array is immediately passed on to the subscriber. That's an array though, I can't think of a way how to pass the values one by one, at the same time fulfilling your specifications. Hope that is enough for your use case.

UPDATE : jsbin here http://jsfiddle.net/zs7ydw6b/2

edited Dec 11 '15 at 01:29

answered Dec 08 '15 at 01:15

user3743222

18,345
5
69
75

This might work, thanks. Although I was thinking about some more reactive-y way. Now trying to make use of `onBackpressureLatest()`. – Dzmitry Lazerka Dec 08 '15 at 09:25
Are you using Java? I was not aware that this operators exists in Rxjs. Not sure either how this is more reactive-y than a regular `scan` but I am always interested to see alternative methods. There is often more than one way to do a given thing with all the operators available. – user3743222 Dec 08 '15 at 15:09
In JS they just named differently, I believe it's `controlled()`. But I'm still not able to make it work, looks like it's the wrong way, so probably your solution will be the way to go. – Dzmitry Lazerka Dec 08 '15 at 18:15
Ok I know this one. The lossless backpressure operators in Rxjs do not offer a replay functionality as far as I know. When an item is requested, it is also consumed, i.e. removed from the buffer. – user3743222 Dec 08 '15 at 18:45
But I still don't understand how to call your example, so that it will hold the array in memory. The `groupedLatestMessages$` is just an observable, it doesn't make the `scan` anything until someone subscribes to it. While my goal is hold in memory only needed messages _before_ anyone subscribes. – Dzmitry Lazerka Dec 10 '15 at 23:13
Probably it will require some internal subscriber, but I cannot figure out how it should look like. Here's a [jsbin](https://jsbin.com/duzuqo/13/edit?js,console) with your code: it doesn't output anything. – Dzmitry Lazerka Dec 10 '15 at 23:17
You can have a look at the jsbin here : jsfiddle.net/zs7ydw6b/2. Discrepancies you observe might come from your use of subjects for the testing. For me it seems to work fine. – user3743222 Dec 11 '15 at 01:30
Ahh, thank you a lot, it really helped. The whole trick was to call empty `.subscribe()` to kick in scan. – Dzmitry Lazerka Dec 11 '15 at 09:37
Your solution can be improved by using Object instead of Array, so it would work for O(1) per message. Note, btw, that if you continue issuing messages after 3 seconds, then `result2` will receive old messages every time, so it keeps holding a message per every user forever. So subscriber needs to keep state of which messages it already seen, and they also consume RAM, if there are lot of distinct users. – Dzmitry Lazerka Dec 11 '15 at 22:05

ReactiveX: Group and Buffer only last item in each group

2 Answers2

Linked