Algorithm behind nginx etag generation

Question

What is the algorithm used to generate etags in Nginx? They look something like "554b73dc-6f0d" now.

Are they generated from timestamp only?

I don't believe they contain an inode (unlike Apache by default)... although I'm having a hard time finding where I found that a long time ago (better for a cache cluster). Are you working in an environment without a useful clock (eg. embedded)? — Cameron Kerr, May 07 '15 at 15:20
There is some related info in https://developer.yahoo.com/performance/rules.html#etags (but not regarding Nginx) — Cameron Kerr, May 07 '15 at 15:26

score 44 · Accepted Answer · edited Aug 30 '19 at 14:32

From the source code: http://lxr.nginx.org/ident?_i=ngx_http_set_etag

1803 ngx_int_t
1804 ngx_http_set_etag(ngx_http_request_t *r)
1805 {
1806     ngx_table_elt_t           *etag;
1807     ngx_http_core_loc_conf_t  *clcf;
1808 
1809     clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);
1810 
1811     if (!clcf->etag) {
1812         return NGX_OK;
1813     }
1814 
1815     etag = ngx_list_push(&r->headers_out.headers);
1816     if (etag == NULL) {
1817         return NGX_ERROR;
1818     }
1819 
1820     etag->hash = 1;
1821     ngx_str_set(&etag->key, "ETag");
1822 
1823     etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3);
1824     if (etag->value.data == NULL) {
1825         etag->hash = 0;
1826         return NGX_ERROR;
1827     }
1828 
1829     etag->value.len = ngx_sprintf(etag->value.data, "\"%xT-%xO\"",
1830                                   r->headers_out.last_modified_time,
1831                                   r->headers_out.content_length_n)
1832                       - etag->value.data;
1833 
1834     r->headers_out.etag = etag;
1835 
1836     return NGX_OK;
1837 }

You can see on lines 1830 and 1831 that the input is the last modified time and the content length.

Compare with [Apache ETags](https://httpd.apache.org/docs/2.4/mod/core.html#fileetag), which are also computed from the modification time and size, but can also be configured to depend on the inode of the file. — Raedwald, Sep 24 '19 at 10:25
More recent source link: https://github.com/nginx/nginx/blob/bfc5b35827903a3c543b58e4562db8b62021c164/src/http/ngx_http_core_module.c#L1675 — aksh1618, Sep 12 '22 at 08:19

score 2 · Answer 2 · answered Nov 24 '18 at 14:14

2

In PHP who will need it.

$pathToFile = '/path/to/file.png';

$lastModified = filemtime($pathToFile);
$length = filesize($pathToFile);

header('ETag: "' . sprintf('%x-%x', $lastModified, $length) . '"');

answered Nov 24 '18 at 14:14

Max_Payne

31
2

3

How is this better than the accepted answer? – RalfFriedl Nov 24 '18 at 14:21
2

@RalfFriedl this answer will be better for PHP programmers, because in the future some PHP programmer like me will look for 3 keywords "nginx", "etag", "alg" and he will find my answer. This perhaps will prevent the creation of duplication of questions. – Max_Payne Nov 24 '18 at 14:34
2

@RalfFriedl it is easier to understand than the nginx source code for someone who is not that familiar with c. – SuperSandro2000 Aug 01 '20 at 18:40
You really shouldn't be generating etags yourself. You should just store whatever value the server gives you. – Kevin Cox Aug 11 '20 at 12:38
@KevinCox this is true for static files, but not for files generated dynamically (ex. using PHP). – Max_Payne Aug 12 '20 at 12:07
I don't know what you mean. If the file is generated dynamically nginx won't generate an etag and there is no need to copy it's algorithm. – Kevin Cox Aug 13 '20 at 10:38
@KevinCox look, it's simple. You can use Nginx to check if a static file exists, right? If not, instead of a 404 error you can tell to Nginx to run a Php file which will handle this case. Php will in turn generate the requested file and send it to the client with a right ETag value. The client will store the file on its machine. Next time Nginx will read the ETag value and will do nothing until the cache is not expired. The ETag should follow Nginx's alg, or it will try to send its ETag again and again. Using this workaround, you can save a lot of heavy computation and space on your server. – Max_Payne Aug 13 '20 at 11:23
That probably isn't harmful, but it is needlessly complicated and coupled. You should just make PHP generate the file with an e-tag, then let nginx handle the caching. nginx is good at caching. – Kevin Cox Aug 13 '20 at 15:46
@KevinCox it seems to me like you don't understand what you're talking about... You just repeated my words without realizing it. This discussion makes no sense any more. – Max_Payne Aug 13 '20 at 19:27
IIUC you are describing PHP generating the file, manually caching it on disk, and then returning the file to nginx with the same etag that nginx would generate, then configuring nginx to serve the cached file by default. I'm describing PHP just generates the content, then you let nginx manage the disk cache. Then PHP doesn't need to worry about writing files at all, you don't need a complicated nginx fallback config and you don't need to worry about syncing etag algorithms. Plus you get other features of nginx caching without re-implementing them all in PHP. – Kevin Cox Aug 13 '20 at 23:21
@KevinCox ahaha, you really thought that I use PHP every time to output some static `/path/to/file.png`? I've repeated several times that PHP is used to generate files **dynamically**. Do you understand what does this mean? It means that a `/path/to/file.png` does not exist physically on the server and you assemble it on the fly, e.g, making a HTTP request or pulling the data from DB. Then you save it to `/path/to/file.png` and of course for the first time send it using PHP with ETag. The next time Nginx will found that `/path/to/file.png` and the caching will work as you described. Good luck. – Max_Payne Aug 14 '20 at 12:18

Algorithm behind nginx etag generation

2 Answers2

Linked