RxJava - get a list, operate on its items, group into list again. toList() waits for onCompleted()

Question

I want to get all unsynchronised items from my database every time they appear there, try to synchronise them to the backend and then write to the database, that they are synchronised. To perform this I make this chain of operations (with RxJava observables):

1. Whenever a change in a database table occurs, get the list of unsynchronised items.
2. For every item in the list - send it to backend.
3. Gather all the items which were successfully sent to backend...
4. ...and write them to the database in a single transaction.

I tried something like this:

newMsgsObservable //from realm database
            .flatMapIterable(new Func1<List<Message>, Iterable<Message>>() { //1
                @Override
                public Iterable<Message> call(List<Message> messages) {
                    return messages;
                }
            })
            .flatMap(new Func1<Message, Observable<Message>>() { //2
                @Override
                public Observable<Message> call(Message message) {
                    return restService.sendMessage(message);
                }
            })
            .toList() //3
            .flatMap(new Func1<List<Message>, Observable<?>>() { //4
                @Override
                public Observable<?> call(List<Message> messages) {
                    for(Message message : messages) {
                        message.setSynchronised(true);
                    }
                    persistence.saveMessages(messages);
                    return null; //yeah, I know, don't bother ;)
                }
            })
            .subscribe();

The problem

newMsgsObservable is always waiting for new changes in database (it never completes) which is good. Unfortunately toList() waits for the observable to complete before it returns the list of items again. I would like a solution which gets a single list from database, performs all the actions on every item on the list, and then gets the same list and returns it to the next observable. And does it every time a new list is available from the initial observable.

Or maybe I'm doing it wrong and I should approach this problem another way?

Answer 1

The main idea is to move your toList() to another level. Thus your code is going to look something like this:

newMsgsObservable.flatMap(new Func1<List<Message>, Observable<List<Message>>>() {
    @Override
    public Observable<List<Message>> call(List<Message> messages) {
        return Observable.from(messages).flatMap(new Func1<Message, Observable<Message>>() {
            @Override
            public Observable<Message> call(Message message) {
                return restService.sendMessage(message);
            }
        }).toList();
    }
})
        .doOnNext(new Action1<List<Message>>() {
            @Override
            public void call(List<Message> messages) {
                for (Message message : messages) {
                    message.setSynchronised(true);
                }
                persistence.saveMessages(messages);
            }
        })
        .subscribe();