best way to save nginx request as a file?

Question

i am looking for a solution to save data sent via http (e.g. as a POST) as quickly as possible (with lowest overhead) via nginx (v1.2.9). i tried the following nginx configuration, but am not seeing any files written in the directory:

server {
  listen 9199;
  location /saveme {
    client_body_in_file_only on;
    client_body_temp_path /tmp/bodies;
  }
}

what am i doing wrong? and/or is there a better way to accomplish this? (the data that is written should ideally be one file per request, and it does not matter if it is fairly "raw" in nature. post-processing of the files will be done via a separate process via a queue.)

You need to setup a cache, responses will then be written to disk. — itpp13, Jun 08 '15 at 20:49
client_body_in_file_only is about BODY only,but you asked for REQUEST in subject, please clarify. Are you sure that you posted any requests with body? — Alexander Altshuler, Jun 09 '15 at 10:22
@alexander my desire is to be able to POST for example a form and have the content of the form written to a file. it need not have all of the data from the request (e.g. headers), but the content would be sufficient, in any format (raw is fine). — Mumonkan, Jun 10 '15 at 17:47

score 6 · Answer 1 · answered May 23 '17 at 08:13

6

This question has already been answered here:

Basically, you need to combine log_format and fastcgi_pass. You can then use the access_log directive for example, to specify where the saved variable should be dumped to.

location = /saveme {
  log_format postdata $request_body;
  access_log  /var/log/nginx/postdata.log  postdata;
  fastcgi_pass php_cgi;
}

It could also work with your method but I think you're missing client_body_buffer_size and `client_max_body_size

answered May 23 '17 at 08:13

Chibueze Opata

9,856
7
42
65

Thank you. I don't have time to check your solution right now, but I think maybe you get the bounty rep points anyway : ) – Steven2163712 May 23 '17 at 08:39

score 1 · Answer 2 · edited Jun 20 '20 at 09:12

Do you mean save cache for HTTP post while someone access and request file and store on hdd rather than memory? I may suggest use proxy_cache_path and proxy_cache. The proxy_cache_path directive sets the path and configuration of the cache, and the proxy_cache directive activates it.

proxy_cache_path /path/to/cache levels=1:2 keys_zone=my_cache:10m max_size=10g
                 inactive=60m use_temp_path=off;
server {
...
    location / {
        proxy_cache my_cache;
        proxy_pass http://my_upstream;
    }
}

The local disk directory for the cache is called /path/to/cache
levels sets up a two‑level directory hierarchy under /path/to/cache/
keys_zone sets up a shared memory zone for storing the cache keys and metadata such as usage timers
max_size sets the upper limit of the size of the cache
inactive specifies how long an item can remain in the cache without being accessed

the proxy_cache directive activates caching of all content that matches the URL of the parent location block (in the example, /). You can also include the proxy_cache directive in a server block; it applies to all location blocks for the server that don’t have their own proxy_cache directive.

best way to save nginx request as a file?

2 Answers2