Update cached rx java observables data (with nested caching!)

Question

I've written a small test that reproduces my problem. I want to nest cached observables and to reflect all changes done to the underlying data structure being done to the ORIGINAL object, but with nested caching this seems to fail.

Is it possible to solve the problem? I have written what I expect under problem...

Test

public static void test()
{
    // --------------------
    // TEST 1 - simple caching
    // --------------------

    List<Data> emptyData1 = getEmptyData()
            .toBlocking()
            .single();

    L.d("TEST 1a - Size: %d == %d", emptyData1.size(), 0);

    // add 1 empty data
    emptyData1.add(new Data(0, false));
    L.d("TEST 1b - Size: %d == %d (Loaded: %b)", emptyData1.size(), 1, emptyData1.get(0).loaded);

    List<Data> emptyData2 = getEmptyData()
            .toBlocking()
            .single();

    L.d("TEST 1c - Size: %d == %d (Loaded: %b)", emptyData2.size(), 1, emptyData2.get(0).loaded);

    // --------------------
    // TEST 2 - nested caching
    // --------------------

    List<Data> loadedData1 = getLoadedData()
            .toBlocking()
            .single();

    L.d("TEST 2a - Size: %d == %d (Loaded: %b)", loadedData1.size(), 1, loadedData1.get(0).loaded);

    // add a second data, this time we add a loaded data
    loadedData1.add(new Data(1, true));

    List<Data> loadedData2 = getLoadedData()
            .toBlocking()
            .single();

    L.d("TEST 2b - Size: %d == %d (Loaded: %b, %b)", loadedData1.size(), 2, loadedData2.get(0).loaded, loadedData2.get(1).loaded);

    // --------------------
    // TEST 3 - test if empty observable is holding the loaded data as well now
    // --------------------

    List<Data> testEmptyStateData = getEmptyData()
            .toBlocking()
            .single();

    L.d("TEST 3a - Size: %d == %d", testEmptyStateData.size(), 2);

    // I don't expect this, but this will happen
    if (testEmptyStateData.size() == 1)
    {
        L.d("TEST 3b1 - Size: %d == %d (Loaded: %b)", testEmptyStateData.size(), 2, testEmptyStateData.get(0).loaded);
    }
    // I expect this, but it won't be true
    if (testEmptyStateData.size() == 2)
    {
        L.d("TEST 3b - Size: %d == %d (Loaded: %b, %b)", testEmptyStateData.size(), 2, testEmptyStateData.get(0).loaded, testEmptyStateData.get(1).loaded);
    }
}

Result

TEST 1a - Size: 0 == 0
TEST 1b - Size: 1 == 1 (Loaded: false)
TEST 1c - Size: 1 == 1 (Loaded: false)

TEST 2a - Size: 1 == 1 (Loaded: true)
TEST 2b - Size: 2 == 2 (Loaded: true, true)

TEST 3a - Size: 1 == 2 // <= UNDESIRED RESULT!!!
TEST 3b1 - Size: 1 == 2 (Loaded: true) // <= at least the object in the list is correct! But I would expect that the empty observalbe would hold 2 items, both loaded! "Test 3b2" should be printed instead

Problem

I would expect that all observables will just pass the ORIGINAL List down the stream and that I always get the original object and that all my changes to this object will be reflected in the base list and therefore at the end, the empty observable should return 2 loaded data items, but that is not true for this nested caching scenario.

What I need

• each operation should only run once! No matter if someone is subscribed to an observable or not! That's whyI decided to use cache()
• observables must be shareable and must be reused

Code

Helper functions and data

private static  Observable<List<Data>> mEmptyObservable = null;
private static  Observable<List<Data>> mLoadedObservable = null;

public static Observable<List<Data>> getEmptyData()
{
    if (mEmptyObservable == null)
    {
        // simple test data
        List<Data> values = new ArrayList<>();
        mEmptyObservable = Observable.just(values)
                // cache and share observable
                .cache().replay().refCount();
    }
    return mEmptyObservable;
}

public static Observable<List<Data>> getLoadedData()
{
    if (mLoadedObservable == null)
    {
        mLoadedObservable = getEmptyData()
                .flatMap(new Func1<List<Data>, Observable<Data>>()
                {
                    @Override
                    public Observable<Data> call(List<Data> datas)
                    {
                        return Observable.from(datas);
                    }
                })
                .map(new Func1<Data, Data>()
                {
                    @Override
                    public Data call(Data data)
                    {
                        data.load();
                        return data;
                    }
                })
                .toList()
                // cache and share observable
                .cache().replay().refCount();
    }
    return mLoadedObservable;
}

Data class

static class Data
{
    int index;
    boolean loaded;

    public Data(int index, boolean processed)
    {
        this.index = index;
        this.loaded = processed;
    }

    public void load()
    {
        if (!loaded)
        {
            // do some have operation... once only per data!
        }
        loaded = true;
    }
}

Answer 1

You're using both cache() and replay(). Try something like

private static  Observable<List<Data>> mEmptyObservable = null;
private static  Observable<List<Data>> mLoadedObservable = null;

public static Observable<List<Data>> getEmptyData()
{
    if (mEmptyObservable == null)
    {
        // simple test data
        List<Data> values = new ArrayList<>();
        mEmptyObservable = Observable.just(values)
                // share and replay observable
                .share().replay();
    }
    return mEmptyObservable;
}

public static Observable<List<Data>> getLoadedData()
{
    if (mLoadedObservable == null)
    {
        mLoadedObservable = getEmptyData()
                .flatMap(new Func1<List<Data>, Observable<Data>>()
                {
                    @Override
                    public Observable<Data> call(List<Data> datas)
                    {
                        return Observable.from(datas);
                    }
                })
                .map(new Func1<Data, Data>()
                {
                    @Override
                    public Data call(Data data)
                    {
                        data.load();
                        return data;
                    }
                })
                .toList()
                // share and replay observable
                .share().replay();
    }
    return mLoadedObservable;
}

There's more information in this answer.

Edit: The real issue however seems to be that you're creating a different list in getLoadedData(). If you want the same list try something like

public static Observable<List<Data>> getLoadedData()
{
    if (mLoadedObservable == null)
    {
        mLoadedObservable = getEmptyData().
                .map(new Func1<List<Data>, List<Data>>()
                {
                    @Override
                    public Data call(List<Data> data)
                    {
                        for (Data item : data) {
                           item.load();
                        }
                        return data;
                    }
                })
                // share and replay observable
                .share().replay();
    }
    return mLoadedObservable;
}