1

Thinking about this more, the question really is: why is Apache putting an in-progress upload in memory instead of on disk? I should be able to upload a file that's larger than the total memory available to the server.

PHP never enters the equation, as the upload never finishes, and Apache dies before it can pass the file to PHP.

Very odd. I've been googling for hours, to no avail.

I have a VPS with 512 MB RAM, and I'm trying to upload files that are between 100-250 MB. Files up to about 100 MB work fine, but anything over and I run into memory problems.

Here's the output from top as I try to upload a 130 MB file, immediately before PID 18367 is killed:

top - 09:43:06 up 6 days,  9:31,  1 user,  load average: 0.00, 0.00, 0.00
Tasks:  51 total,   1 running,  50 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2% us,  0.1% sy,  0.0% ni, 99.8% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:    916144k total,   916144k used,        0k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                       
18367 apache    16   0  721m 393m 2288 S    0 44.0   0:01.90 httpd

Somehow it's using 393 MB of memory for a 130 MB file? (Not to mention, only about 80% of the file had uploaded.) I can't imagine what httpd is doing with all of that space.

Looking in the apache error log, I see:

[Tue Feb 01 09:43:07 2011] [error] (12)Cannot allocate memory: fork: Unable to fork new process
[Tue Feb 01 09:43:17 2011] [notice] child pid 18367 exit signal Segmentation fault (11)

I'm not sure if this a limitation of the software stack (CentOS, Apache, and PHP through mod_fcgid), or a misconfiguration of either Apache or PHP.

I do use the following mod_fcgid directive in my vhost.conf, but to little effect:

MaxRequestInMem 1048576

Are there similar apache configuration values that I ought to tune?

Thanks for your consideration.

JKS
  • 153
  • 1
  • 6
  • Can you show your php code that accepts the upload? Also its seems weird that its dying on fork doesn't it? Sounds like it might be forking infinitely - e.g. a php script that calls itself? – hellomynameisjoel Jun 24 '11 at 04:28

3 Answers3

2

This is a known bug in mod_fcgid, but apparently the mod_fcgid architecture makes it difficult to address in a comprehensive way.

At the moment, the best you can do is to upgrade to mod_fcgid 2.3.7 (or later); it has enhancements that at least release the memory at some point after the file upload has allocated it. My experience so far has been that it can take upwards of half an hour for the memory to be reclaimed, though.

asciiphil
  • 3,086
  • 3
  • 28
  • 53
  • Thanks so much! I actually am still on the same VPS with the same issue (2.5 years later!). I'll try upgrading next time I get a chance. – JKS Apr 29 '13 at 21:10
0

I doubt that's someone's else limitations except the uploader script ('s author(s))…

poige
  • 9,448
  • 2
  • 25
  • 52
  • But the uploader script isn't running into a limit. I'm running out of memory. If I understand you right (?). – JKS Feb 01 '11 at 18:46
  • Do you know how the uploader script works? – poige Feb 01 '11 at 18:48
  • I use [SWFUpload](http://www.swfupload.org/), a little Flash app that allows me to upload files without refreshing the page. When the upload is complete, a custom PHP script handles the file. Is there anything specific I can tell you? – JKS Feb 01 '11 at 19:08
  • What really matters is server-side there, so Flash app is irrelevant. – poige Feb 02 '11 at 02:29
  • Exactly. And the PHP script isn't even called, because the Apache process dies before the hand-off. – JKS Feb 03 '11 at 01:49
0

I have seen similar issues to this before with apache+php (with horde). Try adding to your apache config:

    php_value memory_limit 150M
    php_value post_max_size 150M
    php_value upload_max_filesize 150M

I'm not 100% certain if memory_limit can be set in the apache config. It's possible it needs to be set in php.ini. Optionally, you can also set:

    php_value max_execution_time 180
    php_value max_input_time 180
dialt0ne
  • 3,065
  • 20
  • 27
  • In fact, I already have higher values for all of the options you mention! Gah! Thanks, all the same. – JKS Feb 01 '11 at 23:48
  • "php_value post_max_size 150M" how comes if the topic starter said "I'm trying to upload files that are between 100-250 MB"? – poige Feb 02 '11 at 02:31
  • I was addressing his 130M test file. i.e. get the test to work first, then adjust as needed. – dialt0ne Feb 02 '11 at 16:07