Yield from closure

Question

I would like to yield some data. Problem is that chunk method require closure which do the job. Is there way to yield data like from foreach loop in this situation?

public function yieldRowData(): \Iterator
{
    $this->transactionQuery
        ->getQuery()
        ->chunk(5, function ($collection) { // This is executed for each 5 elements from database
            foreach ($collection as $element) {
                yield $element->getId(); // I want this to be yield from this method (yieldRowData)
            }
        });
}

I tried this, but it will return only last results:

public function yieldRowData(): \Iterator
{
    $results = [];

    $this->transactionQuery
        ->getQuery()
        ->chunk(5, function(Collection $transactions) use (&$results) {
            $results = $transactions;
        });

    foreach($results as $transactionEntity) {
        yield $transactionEntity;
    }
}

Check: https://stackoverflow.com/questions/29744661/using-yield-in-callback — Alexey Chuhrov, Mar 07 '18 at 12:01
I did, but this doesn't work for me. Can you provide some code? I posted my attempts to implement that. — imclickingmaniac, Mar 07 '18 at 12:19
Is the `chunk` method part of some third-party library / framework? I think what you need is some version of that method which itself returns an iterator, or some similar object where you explicitly ask for the next chunk, rather than having it delivered to a callback. — IMSoP, Mar 07 '18 at 12:28

score 0 · Answer 1 · answered Jul 11 '19 at 08:42

You didn't mention explicitly whether chunk() method (or whatever) returns the yielded data from within the closure or not; but I guess it won't.

If it returns (i.e. it's a generator function), then yield from is the best choice:

// We are inside the method
yield from $query->chunk(5, function ($collection) {
    foreach ($collection as $element) {
        yield $element->getId();
    }
});

However, if it is not a generator function, then you have to use other solutions. Probably, as others mentioned, you don't want to save the results inside a variable and return it, as it wastes memory (especially on large data sizes).

But there is another solution that may work. If you are lucky, the code in chunk() won't be complicated, so you can extend from its class and rewrite the chunk() method as a generator. You may not be able to do that, but usually it is possible and not really hard or time-consuming.

Oh my... you are answering/questioning problem I had over 1 year ago :D As I remember return value was not generator. Everything was done inside library and specified function. I did found some solution for this, but I don't even have access to that project anymore to check how it was done or omitted. — imclickingmaniac, Jul 12 '19 at 12:37
@imclickingmaniac Oh, sorry for the one-year latency. Others may be helped by this, however. — MAChitgarha, Jul 12 '19 at 12:52

ChristianM · Answer 2 · 2018-03-08T09:26:18.543

-2

Try this it should resemble your first try more closely while actually yielding something sensible:

public function yieldRowData(): \Iterator
{
    $results = [];

    $this->transactionQuery
        ->getQuery()
        ->chunk(5, function(Collection $collection) use (&$results) {
            foreach ($collection as $element) {
                $results[] = $element->getId();
            }
        });

    foreach($results as $transactionEntity) {
        yield $transactionEntity;
    }
}

Adding alternative idea as to the comment below: Pass in the closure to yieldRowData() in order to use the memory efficiency of chunk:

public function workRowData(\closure $rowDataWorker)
{
    $this->transactionQuery
        ->getQuery()
        ->chunk(5, $rowDataWorker);
}

$rowDataWorker could be bound to a variable that should contain some end result or can use $this to access instance variables.

edited Mar 08 '18 at 09:26

answered Mar 07 '18 at 14:55

ChristianM

1,793
12
23

`->getId()` was just example. In fact I need quite more data. Your example will work, but then `chunk` have no sense, as you build whole data on memory. In such situation `yield` is also useless, as you already set full data in array, so you can return it instead of yielding. Such code would not save memory at all. Thank you. – imclickingmaniac Mar 08 '18 at 08:32
That is indeed true, what you could do is pass in the closure that operates on the data. `yieldRowData(\closure $rowDataWorker)` if that helps you. – ChristianM Mar 08 '18 at 08:57
Your second example introduces a wrapper but no extra behaviour; it's the definition of `$rowDataWorker` that the OP is struggling with, not where to pass it. – IMSoP Mar 08 '18 at 09:45
The OP has not given any indication how the data is used, so I am simply giving another way to think about the problem as the original problem is not solvable in the way requested. :) – ChristianM Mar 08 '18 at 09:48
1

@ChristianM Yeah, I'm just pointing out that it doesn't really reframe the problem at all, it just gives the single statement `$this->transactionQuery->getQuery()->chunk(5,$foo)` a function name so you can shorten it to `something($foo)`. It might as well be called `function thisTransactionQueryChunk5($rowDataWorker)`; the rest of the code around it will look exactly the same, and work or not in exactly the same way. – IMSoP Mar 08 '18 at 16:32
1

This negates the benefits of a generator entirely. You are first loading the entire result into PHP memory and then yielding each value. In a context like database results a generator allows you to process each row individually without loading the entire result into memory. – JamesHalsall Sep 05 '18 at 16:35

Yield from closure

2 Answers2