I'm trying to develop a Source
that turns a set of cassandra partitions into an Akka stream like this (I execute multiple cassandra queries through a bound statement, one for each partition and then I mapConcat the result into an iterable of DTOs):
Source<Result<JournalEntry>, NotUsed> src = Source.unfoldAsync(queryBag, s -> {
BoundStatement bound = getBoundStatement(queryBag);
// get out data from cassandra:
Future<Result<JournalEntry>> page =
Future.apply(() -> journalMapper.map(session.execute(bound)), ec);
Future<Optional<Pair<QueryBag,Result<JournalEntry>>>> next = page.map(r -> {
Optional<Pair<QueryBag,Result<JournalEntry>>> opt;
if (r.isExhausted()) {
opt = Optional.empty();
} else {
opt = Optional.of(Pair.apply(queryBag.incrementPartitionId(), r));
}
return opt;
}, ec);
return FutureConverters.toJava(next);
});
Source<JournalEntry, NotUsed> concat = src.mapConcat(res -> () -> res.iterator());
// materialize concat and consume the stream
Is this a safe practice? In particular, I'm concerned about generating the iterable through mapConcat((res) -> () -> res.iterator())
, as multiple calls of iterator()
method of the resulting Iterable
would get the same instance of iterator.
Besides, is this a sensible approach to iterate across different cassandra partitions or would it be more appropriate to use a GraphStage
?