Joining flattened data from Firebase Realtime Database using RxJava

Question

Joining flattened data is a common use case also described in the documentation. But the documentation shows a simple example which is not real-time, it doesn't react to changes. I'm looking for a more robust implementation. I think RxJava is ideal for this.

Consider following Firebase structure:

{
  "messages": {
    "group_id_1": {
      "message_id_1": {
        "text": "Hello",
        "author": "uid_1"
      }
    }
  },
  "users": {
    "uid_1": {
      "name": "David"
    }
  },
  "rooms": {
    "room_id_1": {
      "name": "General",
      "members": {
        "uid_1": true
      }
    }
  }
}

I see two use-cases here:

1. Get list of messages in a group with author names
   • I imagine I would get Observable<Message> and when I subscribe to it, dependencies (users for those messages) will be subscribed as well in some cache. When I'm showing the messages, I can get author's names from the cache.
   • It's also real-time - if author name changes, the observable emits changed Message.
   • When I unsubscribe to the observable, also dependencies unsubscribes.
2. Get a list of room members with their names
   • I imagine I would get Observable<User> and when I subscribe to it, it will first subscribe to room's members and then to individual users.
   • It's real-time - if room members change, I get notified about that.
   • When I unsubscribe to the observable, also dependency unsubscribes.

Do you know about library/solution which could do that?

Or would you use it if I created one?

Answer 1

I was going to pose a variation of this question but seemed like it might be better to build on top of this one...I'll describe what is hopefully at least partially the answer to above question but also then a shortcoming I'm seeing.

Using above data model we might have something like following to create RxJava wrapper around firebase query to get list of member keys for particular room and for getting details for particular member (note use of onCompleted() in subscriber.getMemberInfo...more on that later!).

public Observable<String> getRoomMembers(String roomId) {
    return Observable.create(subscriber -> {
        databaseReference.child(roomId + "/members").addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                for (DataSnapshot childSnapshot: dataSnapshot.getChildren()) {
                    String userId = childSnapshot.getKey()
                    subscriber.onNext(userId);
                }
                subscriber.onCompleted();
            }

            @Override
            public void onCancelled(DatabaseError databaseError) {
            }
        });
    });
}

public Observable<Member> getMemberInfo(String memberId) {
    return Observable.create(subscriber -> {
        databaseReference.child(memberId).addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                Member member = dataSnapshot.getValue(Member.class);
                subscriber.onNext(member);
                subscriber.onCompleted();
            }

            @Override
            public void onCancelled(DatabaseError databaseError) {
            }
        });
    });
}

What we can then do use something like following to get list of Members for particular room (have added isActive property to Member to show how we also filter results we get).

    getRoomMembers(roomId)
            .flatMap(memberId -> getMemberInfo(memberId))
            .filter(Member::isActive)
            .toList()
            .subscribe(members -> {

            });

So, the above works up to a point. The issue is that I had to call subscriber.onCompleted() in getMemberInfo for the above call to flatMap to work....which then means that any subsequent changes to Member data isn't triggering update in the above subscription. Am relatively new to RxJava and Firebase so might be missing something obvious.

Answer 2

I eventually solved it with these two methods (in Kotlin, Java is similar, just more verbose):

fun <A, B> Observable<List<A>>.mapSubQueries(subQuery: (A) -> Observable<B>): Observable<List<Observable<B>>> {
    return this.flatMap {
        if (it.isNotEmpty()) {
            return@flatMap Observable.from(it).map { subQuery(it) }.toList()
        } else {
            return@flatMap Observable.just(listOf<Observable<B>>())
        }
    }
}

@Suppress("UNCHECKED_CAST")
fun <T> Observable<List<Observable<T>>>.joinSubQueries(): Observable<List<T>> {
    return this.flatMap {
        if (it.isNotEmpty()) {
            return@flatMap Observable.combineLatest(it, {
                val list = mutableListOf<T>()
                it.forEach {
                    list.add(it as T)
                }
                list
            })
        } else {
            return@flatMap Observable.just(listOf<T>())
        }
    }
}

To get users in all messages, I can use it like this:

fun usersInMessages(roomId): Observable<List<User>> {
    return DatabaseRead.messages(roomId)
            .mapSubQueries { DatabaseRead.user(it.getAuthor()) }
            .joinSubQueries()
}

I decided that it's better to keep this code in my codebase and modify it slightly for various use-cases. Making it a library would make it less flexible. The main point is always use Observable.combineLatest(). Many other Rx parameters are useless, because they require onComplete() call and here I deal with infinite Observables.