4

The PHP 7.4 (on Windows) code is:

<?php
// index.php

$postBodyResource = fopen("php://input", 'rb');
while (!feof($postBodyResource)) {
    $data = fread($postBodyResource, 5 * 1024);
}
fclose($postBodyResource);

The key part of the Apache 2.4 (on Windows) config is

Alias "/api" "${SRVROOT}/htdocs/Api"
<Directory "${SRVROOT}/htdocs/Api">
    RewriteEngine On 
    RewriteCond %{REQUEST_FILEaNAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^ index.php [QSA,L]

    Script POST "index.php"
    Script PUT "index.php"

    php_value enable_post_data_reading Off
</Directory>

If I do a HTTP POST (Content-Type: application/octet-stream) at http://localhost/api with a body larger than 8 kB, PHP creates a temp file at sys_temp_dir that stores the whole content of the stream php://input while doing fread(). Why?

Do we know a method to avoid this behaviour? I want to read the stream without any disk/SSD access – just parts in memory. Finally, I want to get the SHA3-512 hash value of the HTTP body, that can be very large. Therefore writing to temp is a critical bottleneck.

raubvogel
  • 126
  • 1
  • 8
  • Are you sure it is PHP which is writing the file but not Apache or nginx (if used)? – Jared Mar 21 '22 at 00:55
  • could it be the program used to make the request? postman makes temp files – chiliNUT Mar 21 '22 at 01:18
  • Quick guess, PHP is backing the memory, which is often limited, with something with more room. A common PHP config is 128 or 256 MB per request. If most of that is used by PHP itself, there’s no room for requests. Are these tmp files causing you problems? – Chris Haas Mar 21 '22 at 03:40
  • @Jared, the temp files are named like `php2526.tmp` – so, yes. While stepping with the debugger through the code, one can clearly see that the temp file is created and updated after `fread()`. – raubvogel Mar 21 '22 at 07:28
  • @chiliNUT, the request is made by a java `HttpURLConnection`. So, I do not suspect the client to create the temp file. – raubvogel Mar 21 '22 at 07:30
  • @ChrisHaas, I have the same guess. But `php://input` is a "read-only stream that allows you to read raw data from the request body" [https://www.php.net/manual/en/wrappers.php.php](php doc). So, I guess, there is no need to store this stream in a temp file – that’s up to the consumer of `php://input` (like in my case there is no need to store the stream at all). In my case temp files are a huge problem: I will receive many streams in parallel having a huge content. Therefore the disk writes tons of data for no sense … – raubvogel Mar 21 '22 at 11:44
  • So, if this _isn't_ backed by disk, they'd need to be backed my RAM them. Do you have enough RAM for the entire request + application, multiplied by your expected number of connections? – Chris Haas Mar 21 '22 at 13:35
  • @ChrisHaas, the point is: there is no need to back the _entire_ stream data. One of the advantages to uses streams is that you do not need to know _all_ data at a certain point of time. In my case a hash function that supports streaming only needs to know some part (e. g. 1024 bytes) of the data at a certain point of time (the “incremented” final hash value is calculated at end of stream). – raubvogel Mar 21 '22 at 19:00
  • 5
    The [temporary file creation appears to be built into PHP](https://github.com/php/php-src/blob/01b3fc03c30c6cb85038250bb5640be3a09c6a32/ext/standard/php_fopen_wrapper.c#L231) and according to [this test](https://github.com/php/php-src/blob/423c70fb4d79b7831b1db41ea217c8e1afd5cf8e/tests/basic/enable_post_data_reading_06.phpt), it appears to be locked at 8k. Looking at that test, it appears backing the input with a file stream is done in order to support multiple consumers that are seeking at different times. – Chris Haas Mar 21 '22 at 19:44
  • @ChrisHaas, thanks for the research! Is it worth to submit a bug/enhancement report? At the moment I do not see any work around. – raubvogel Mar 21 '22 at 21:45
  • 1
    I honestly don’t think it is a bug, but it is behavior that appears to be undocumented (as far as I can find), so I think it is worth asking about, at least. Maybe there’s a setting somewhere, for all I know. You could file a bug, or ask on the [internals list](https://news-web.php.net/php.internals) – Chris Haas Mar 22 '22 at 00:03
  • I just created an issue for php-src: [https://github.com/php/php-src/issues/8239](https://github.com/php/php-src/issues/8239). – raubvogel Mar 25 '22 at 09:13

0 Answers0