I understand that we can do checkAndDelete on BigTable using the HBase API. I wonder if there is a way for me to submit a list of checkAndDelete so that I don't need to do them one by one? I mostly worry about the performance about non-batching processing...
Thanks!
The HBase API doesn't offer a bulk option for checkAnd* methods. The HBase API is backed by a gRPC protobuf API, which is natively asynchronous. You can use the underlying API to construct a high throughput application.
A WORD OF WARNING: this is mostly undocumented in our public facing documentation, but the source code should provide enough instruction on how to construct your application. There are active users of this API, including Apache Beam's BigtableIO. TODO: document this feature, feel free to comment or vote on the feature.
At a high level, here's what you'd have to do:
AbstractBigtableConnection btConn = (AbstractBigtableConnection) connection;
// BigtableSession can also be constructed by passing in a BigtableOptions.
BigtableSession session = btConn.getSession();
// AsyncExecutor encapsulates the notion of a "batch" of RPCs
// that need to be completed as a unit.
AsyncExecutor executor = session.createAsyncExecutor();
// This listenable future will inform you when the request is done,
// and if there were any exceptions.
// You can attach a FutureCallback
ListenableFuture<CheckAndMutateRowResponse> response =
executor.checkAndMutateRowAsync(checkAndMutateRowRequest);
// do more async work
// Make sure all of the RPCs are complete.
asyncExecutor.flush();
For reference, here's more information about ListenableFuture and FutureCallback. You can also dig into BigtableBufferedMutator for some suggestions on how to handle exceptions.
You can create a CheckAndMutateRowRequest by following the implementation of BigtableTable.checkAndDelete.
