3

Using string.format() (which supposedly defers to C's sprintf())) to format a number in LuaJIT rounds differently than every other Lua interpreter I've tried:

$ lua -v
Lua 5.4.1  Copyright (C) 1994-2020 Lua.org, PUC-Rio
$ lua -e 'print(string.format("%.4f", 32.90625))'
32.9062
$ luajit -v
LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2020 Mike Pall. http://luajit.org/
$ luajit -e 'print(string.format("%.4f", 32.90625))'
32.9063

Why does this happen and how can I‌ make it stop? I'd like to tell LuaJIT to round the same direction every other Lua interpreter does.

Caleb
  • 5,084
  • 1
  • 46
  • 65
  • Looks like they have a custom implementation. Followed function calls to https://github.com/LuaJIT/LuaJIT/blob/1d8b747c161db457e032a023ebbff511f5de5ec2/src/lj_strfmt_num.c#L269 then quickly gave up :) – EDD Oct 27 '20 at 20:48
  • `I'd like to tell LuaJIT to round` - Implement your own `string.format` in LuaJIT FFI by invoking standard function `sprintf` as vanilla Lua does. – Egor Skriptunoff Oct 27 '20 at 20:52
  • In LuaJIT ```do local lambda=function(f) return f-0.00001 end print(string.format('%.4f',lambda(32.90625))) end``` – koyaanisqatsi Oct 27 '20 at 22:24
  • @koyaanisqatsi That's roughly [what I hacked together](https://github.com/sile-typesetter/sile/commit/81620829f55a46f73f7281b9172c0056438fba2d), with a few caveats. First, it doesn't work to coerce LuaJIT like that because it doesn't handle all precisions the same way, so I coerced all other Luas to be like LuaJIT instead. Also I had to account for not making 0 a negative and rounding negative floats the same direction. Still the result feels dirty as the dregs of an oil pan—hence why I didn't self answer. Surely there is a better way. – Caleb Oct 28 '20 at 22:09
  • Hey, a helper function like lambda is used by the giants of science. Einstein do it for example. Therefore i choose this name for it. A more clean way could be to add the math functions as ```__index``` metamethod and write a more precise lambda function to add ```math.f_floor()``` ? Try to see what i mean: ```do float=32.90625 debug.setmetatable(float,{__index=math}) print(float:floor()) end``` – koyaanisqatsi Oct 29 '20 at 09:42
  • yea, LuaJIT seems to round if precision is less than 13 `else if (prec < 13) { /* Precision is sufficiently low as to maybe require rounding. */` You could floor it, at the required digit, to make them return similar results `print( string.format( "%.4f", math.floor( 32.90625 *10000 ) /10000 ) )` – Doyousketch2 Oct 29 '20 at 12:29
  • 1 followed by four 0's shifts the decimal over four places, floor() drops everything after that, then divide by the same amount to recover your original decimal location. – Doyousketch2 Oct 29 '20 at 12:34
  • @Wolf My personal website is offline for a while for personal reasons. It will come back when it comes back. In the mean time you can see from the SO interface that this question is still open (even if not relevant to me, others have upvoted so you can see it's relevant to others too), and as for which direction I want the round to happen that's already spelled out in the question. – Caleb Jun 16 '21 at 13:36
  • @Wolf Check the last sentence, the examples are just a sample. I want to make LuaJIT behave the same way as other Lua implementations do. – Caleb Jun 17 '21 at 09:11
  • @Caleb Interestingly you seem to be absolutely sure that the question has perfect wording. I'm really sorry for the time I wasted here. Too bad! – Wolf Jun 17 '21 at 15:43
  • @Wolf I'm not saying it's perfect, but the info is there and you know what the question is now even if you missed it the first time. If you don't think it is a clear question either downvote it, edit it to make it clear, or maybe just leave it alone for somebody that recognizes the problem and knows a solution to answer. – Caleb Jun 18 '21 at 20:56

0 Answers0