2

I'm creating about 150 nodes programmatically and running into 'out of memory' errors when doing it all in a single request. (I have a menu callback that generates the nodes and calls node_save() on them.)

Example:

for($i=0; $i<150; $i++) {
    $node = new stdClass(); 
    $node->title="Foo $i";
    $node->field_myfield[0]['value'] = "Bar $i";
    ...
    node_save($node);
}

I've heard of BatchAPI, but never used it. Is that the right tool to get around this? The docs talk about timeouts, but not memory issues. Is there something simpler that I might be missing?

apaderno
  • 28,547
  • 16
  • 75
  • 90
sprugman
  • 19,351
  • 35
  • 110
  • 163
  • are you using node_load() to get information on other nodes? – gapple Apr 27 '10 at 22:18
  • No, at the moment, these are completely built from scratch. I'll add a bit of example code above. – sprugman Apr 27 '10 at 22:42
  • This shouldn't be too much a problem. How much RAM is your electronic computer device running? – Rimian Apr 28 '10 at 06:25
  • Turns out I was doing `dpm($node)` in a hook_node() function elsewhere, so maybe that was causing the problem. When I commented that out, I made it through without BatchAPI. – sprugman Apr 28 '10 at 23:26
  • Yes, dpm() needs to store the displayed data in memory and holds it until page rendering. On modern PHP version, your code allow memory to be reclaimed by PHP's garbage collector so you should not encounter 'out of memory' error. However, using the Batch API would also prevent likely timeouts if the DB gets a bit overloaded. – Pierre Buyle Jun 04 '13 at 15:47

2 Answers2

2

Yes, Batch API can solve this problem. It will break up your memory usage into separate HTTP requests, each with access to your full memory limit.

Scott Reynen
  • 3,530
  • 18
  • 17
0

have you ever used Views Bulk Operations? (http://drupal.org/project/views_bulk_operations) it comes with a bundled view displayed at admin/content/node2 you may edit that to enable a "Run PHP code" action, as well as turn on Batch API. it is the easiest way to programmaticaly modify nodes.

however, since you are creating the nodes, you should just unset the $node at the end of the instruction, and it should tone down your memory usage. try: 

  for($i=0; $i 150; $i++) {
    $node = new stdClass(); 
    $node->title="Foo $i";
    $node->field_myfield[0]['value'] = "Bar $i";
    ...
    node_save($node);
    unset($node);
  }

}

Capi Etheriel
  • 3,542
  • 28
  • 49
  • well, my actual code is more like `for (1-150) { mymod_create_node(); }` so the memory should be released each time the function exits in theory. – sprugman Apr 28 '10 at 23:28