Intermittently echo out data with long-running PHP script

Question

I have a PHP script that makes a bunch of cURL requests. After each cURL request, I want to echo out some data, but presently, data only gets echoed out after every 5-10 cURL requests.

I've tried using ob_flush and flush, but it doesn't seem to make a difference. The following is the basic flow of my script:

<?php

  header('Content-Type: text/html; charset=UTF-8');

  set_time_limit(0);

  ob_start();

  $arr = array(); // Lots of strings in this array

  foreach ($arr as $elem) {

    // Use $elem to make cURL request and return HTML.
    // Run regexes on returned HTML.

    echo '<pre>';

    print_r($matches[1]);

    print_r($matches[2]);

    echo '</pre>';

    ob_flush();

    flush();

  }

Is there anything I can do to force the script to output the echoed/print_r'ed data after each iteration of the foreach loop?

Thank you very much.

It was disabled, but I just uncommented the line `LoadModule deflate_module modules/mod_deflate.so` in the `httpd.conf` file and restarted the Apache server, but it didn't seem to make a difference. — HartleySan, Dec 01 '13 at 19:49
I should mention that if you run a script like this from the command line, it'll output all of the data you want when you ask it to regardless of the amount of data being echoed. Also, you don't need to use the ob_ functions. — HartleySan, Jun 19 '17 at 12:39

Drew · Answer 1 · 2013-12-01T21:45:43.873

1

You need to move the ob_start() inside the loop, as:

<?php

  header('Content-Type: text/html; charset=UTF-8');

  set_time_limit(0);

  $arr = array(); // Lots of strings in this array

  foreach ($arr as $elem) {

    ob_start();

    // Use $elem to make cURL request and return HTML.
    // Run regexes on returned HTML.

    echo '<pre>';

    print_r($matches[1]);

    print_r($matches[2]);

    echo '</pre>';

    ob_end_flush();

    flush();

  }

Think of the Output Buffer (ob_*) functions as push and pop on a stack. You specify where you want to start recording by pushing a buffer onto the stack (ob_start()) and then when you want to output, you pop the buffer off the stack and do something with the result (ob_flush(), ob_get_*(), etc). Each ob_start() must have a matching buffer end function.

You'll also want to use ob_end_flush() instead of ob_flush() as I don't think you want keep the buffer after each run.

edited Dec 01 '13 at 21:45

answered Dec 01 '13 at 19:47

Drew

1,687
5
25
46

I tried this, but it didn't do what I wanted. In fact, when I do this, the script waits for all of the cURL requests to finish before outputting everything at once. – HartleySan Dec 01 '13 at 19:57
I added a note about `ob_flush()` vs `ob_end_flush()`, that didn't work either? – Drew Dec 01 '13 at 19:58
I just tried `ob_end_flush` instead, and it did make a slight difference, but the data is still only output every 10 cURL requests or so. There are a couple of things I'm wondering though: 1) I'm only outputting about 500 characters worth of text per cURL request. Is the small amount of data perhaps causing PHP to combine requests? 2) I tried enabling mod_deflate, as recommended by user1281385, but that doesn't seem to make a difference. Do I need to be concerned about that? – HartleySan Dec 01 '13 at 20:02
`mod_deflate` will mean that for all intents and purposes the `ob_*` functions are ignored, because the server end has to collect the entire output and compress it before it sends it. But if you're getting incremental results, it's probably not mod_default.Can you put an `echo microtime();` before `ob_start()`, before the curl stuff, and before `ob_end_flush()`? – Drew Dec 01 '13 at 20:09
Actually, scrap the microtime. Maybe I was wrong, and you need http://us2.php.net/manual/en/function.flush.php. Might be worth adding some line endings. – Drew Dec 01 '13 at 20:14
Adding `flush()` after `ob_end_flush()` did speed things up slightly, but only slightly. Now, data is output every 7-8 cURL requests instead of every 10 or so. – HartleySan Dec 01 '13 at 20:21
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/42283/discussion-between-drew-and-hartleysan) – Drew Dec 01 '13 at 20:53

score 0 · Answer 2 · answered Dec 01 '13 at 20:21

0

Try using this at start:

apache_setenv('no-gzip', 1);
ini_set('output_buffering', 0);
ini_set('zlib.output_compression', 0);
ini_set('implicit_flush', 1);

And then do the stuff you already did.

answered Dec 01 '13 at 20:21

nl-x

11,762
7
33
61

Thanks for the answer. Unfortunately, this didn't speed anything up at all. I'm still only getting output every 7-8 requests. – HartleySan Dec 01 '13 at 20:35

Intermittently echo out data with long-running PHP script

2 Answers2