0

Assume below code which processes data from a paginatedAPI (external).

Flowable<Data> process = Flowable.generate(() -> new State(), 
new BiConsumer<State, Emitter<Data>>() {
void accept() {
                           
    //get data from upstream service
    //This calls dataEmitter.onNext() internally
    pageId = paginatedAPI.get(pageId, dataEmitter);

    //process
    ...

    //update state
    state.updatePageId(pageId);
}
}).subscribeOn(Schedulers.from(executor));

Now, since this is created from .generate, accept will be called only when subscriber is ready for next data. I have full control on what I can add to state, but I can't change paginatedAPI

Requirement:

  1. After a time T from subscription,
    a) Iterate through all pages without sending them to subscriber and call paginatedAPI.close()
    b) Provide subscriber with data from paginatedAPI.close()

  2. If the subscriber disconnects before time T, then
    a) Iterate through all pages without sending them to subscriber and call paginatedAPI.close()

I don't understand how to add the concept of time from subscription in controlling the flowable logic.
Also, accept can only call onNext atmost once. Now how can I finish through the paginatedAPI by calling onNext multiple times.

Edit: Added details on emitter and internal onNext call in paginatedAPi.get(pageId, dataEmitter);

gGwP
  • 35
  • 1
  • 6
  • I don't understand your requirements. How do you iterate over all pages? You can generate a list of things, `onNext` them and flatten the items after generate with `flatMapIterable(v -> v)`. – akarnokd Oct 15 '22 at 08:03
  • Sorry I mis-coded the biConsumer in the question. The `paginatedApi.get(pageId, dataEmitter)` internally calls `onNext` on biConsumer's emitter. – gGwP Oct 18 '22 at 19:44
  • This limits me from calling `paginatedApi.get(pageId, dataEmitter)` multiple times to iterate through all pages. I want to know if that iteration can be somehow done outside `.generate` and still be able to return data from `paginatedApi.close()` to subscriber. I'm aware of the timeout operators, but they all check time from last item. I found it hard to implement the overall logic with the operators available. – gGwP Oct 18 '22 at 19:46
  • The iteration can't be done inside `.generate` since it limits atmost once `onNext` operation on the emitter. – gGwP Oct 18 '22 at 19:47
  • You can have your own `Emitter` instance with methods doing nothing so give that to `paginatedApi.get`. – akarnokd Oct 19 '22 at 06:27

0 Answers0