Multi collectItems how to early terminate and return the collected items

Question

In my quarkus service, i am building a custom accumulator to consume a Multi. The stream can potentially be infinite, and I wonder how to early terminate and collect the results when sufficient data has been accumulated?

Below are my prototype:

Multi<Data> sortedStream = getStream();
return this.sortedStream.collectItems().in(
    LinkedList::new,
    new BiConsumer<LinkedList<Coverage>, Data>() {
        @Override
        public void accept(LinkedList<Coverage> coverages, Data incoming) {
        if (coverages.isEmpty()) {
            coverages.add(new Coverage(incoming));
            return;
        }

        if (enough(coverages)) {
            // Question: How to early terminate and collect coverage downstream?
        }

        Coverage last = coverages.getLast();
        if (worthAdd(last, incoming)) {
            coverages.add(new Coverage(incoming));
        } else {
            return;
        }

        }
    }
)

I note that the link says _Don’t collect items from infinite streams or you will likely end with an out-of-memory failure!_ so if you limit the stream somehow upstream so it'll eventually stop, couldn't you then just return; when you have sufficient coverages ?? (i.e. skip creating and evaluating new coverages at that point??). — Mr R, Mar 11 '21 at 02:21
Memory is not an issue in my case, because items are checked before adding to the results. I am pretty confident that the list won't exceed a few hundred items. — GeauxEric, Mar 11 '21 at 02:39
Couldn't you also do the upstream kind of like what you've doing - following this pattern `upstream.select().where(f).collect().in(f)` and f is something that implements `BiConsumer` and `Function` [but is stateful - after you've had enough coverages you make sure that the function drops any further elements ...] — Mr R, Mar 11 '21 at 02:45
if you only have a few hundred items don't worry about filtering anything out, just return whenvever you've had enough (it'll be called but quickly complete). — Mr R, Mar 11 '21 at 02:46
The aforementioned list is the final result. I may still need to check a lot more items from the stream. Besides, whether to terminate depends on the content of the list, not on the item. I would like to terminate, instead of dropping data. I am seeking a more elegant way to handle such issue in `Multi`. — GeauxEric, Mar 11 '21 at 03:00
Sorry if it wasn't clear .. at the point where your Question is, just return. — Mr R, Mar 11 '21 at 03:01

score 1 · Answer 1 · answered Mar 11 '21 at 21:05

According to the official team (https://github.com/smallrye/smallrye-mutiny/issues/501), collect collects all the items until a terminal event. I managed to tweak my algorithm to come up with a way to check each item upstream and early terminate using multi.select().first API.

Multi collectItems how to early terminate and return the collected items

1 Answers1