2

I have the following code that upserts a collection of pojos( to be turned to json documents) into couchbase using Observables:

    public <T> long  batchUpsert(Iterable<T> items, Function<T, JsonDocument> docCreator, Bucket couchbaseBucket) {

    AtomicLong counter = new AtomicLong();

    AsyncBucket asyncBucket = couchbaseBucket.async();
    Observable<JsonDocument> observableFromDocs =
            Observable
                    .from(items)
                    .map(elem -> docCreator.apply(elem))
                    .filter(elem -> elem!=null)//skip creating problematic docs. logging their info for troubleshooting
                    .flatMap(elem -> upsertDocument(elem, asyncBucket))
                    .retryWhen(
                            RetryBuilder.anyOf(BackpressureException.class, Exception.class)
                                    .doOnRetry((Integer integer, Throwable throwable, Long aLong, TimeUnit timeUnit) ->
                                            log.error("Retrying load. Attempt {} For exception {}", integer,throwable.toString())
                                    )
                                    .delay(Delay.exponential(TimeUnit.MILLISECONDS, RETRY_DELAY_CEILING))
                                    .max(MAX_RETRIES)
                                    .build()
                    );

    observableFromDocs.subscribe(
            (elem)-> {},
            elem -> log.error("Document insertion failure", elem),
            () -> {counter.incrementAndGet();log.debug("Completed ASYNC load ");});

    return counter.get();
}

This code runs fine, the documents get created and the async call is made to upsert but no document is upserted to couchbase, it fails silently, and no exceptions are logged, almost as if the thread dies internally, can someone kindly point what am I doing wrong? I am about to pull my hairs... :)

I now verified it fails for collections of 1 item, can someone tell me why that's happening?

Carlos Luis
  • 213
  • 2
  • 17

1 Answers1

0

I think your code can not wait all async operations finished so this situation is causing the problem Official sdk dock have a example for batch operations and the example say that wait last operation and call block

kakashi hatake
  • 1,175
  • 2
  • 10
  • 18